[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:15:41 dist-test-slave-s11t krb5kdc[15672](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:15:41 dist-test-slave-s11t krb5kdc[15672](info): set up 2 sockets
Aug 09 02:15:41 dist-test-slave-s11t krb5kdc[15672](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260809 02:15:43.070676 15662 mini_kdc.cc:121] Time spent starting KDC: real 1.244s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Aug 09 02:15:43 dist-test-slave-s11t krb5kdc[15672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241743, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:15:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:15:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.75.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab.
Entry for principal kudu/127.15.75.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab.
I20260809 02:15:43.191813 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:40021
--webserver_interface=127.15.75.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab
--principal=kudu/127.15.75.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:36643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.75.190:40021
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:15:43.311487 15698 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:15:43.311779 15698 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:15:43.311861 15698 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:15:43.315817 15698 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:15:43.315901 15698 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:15:43.315940 15698 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:15:43.316001 15698 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:15:43.316041 15698 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:15:43.321192 15698 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:36643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.75.190:40021
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab
--principal=kudu/127.15.75.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:40021
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.75.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15698
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:15:43.322201 15698 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:15:43.323221 15698 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:15:43.332134 15707 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:15:43.332137 15710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:15:43.332216 15698 server_base.cc:1034] running on GCE node
W20260809 02:15:43.332190 15704 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:15:43.333086 15698 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:15:43.334203 15698 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:15:43.335386 15698 hybrid_clock.cc:648] HybridClock initialized: now 1786241743335334 us; error 71 us; skew 500 ppm
Aug 09 02:15:43 dist-test-slave-s11t krb5kdc[15672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241743, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:15:43.338423 15698 init.cc:377] Logged in from keytab as kudu/127.15.75.190@KRBTEST.COM (short username kudu)
I20260809 02:15:43.339399 15698 webserver.cc:460] Webserver started at http://127.15.75.190:34099/ using document root <none> and password file <none>
I20260809 02:15:43.339975 15698 fs_manager.cc:359] Metadata directory not provided
I20260809 02:15:43.340080 15698 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:15:43.340327 15698 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:15:43.342490 15698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/minicluster-data/master-0/data/instance:
uuid: "f52e52b03ac6427889a60294bd825a1b"
format_stamp: "Formatted at 2026-08-09 02:15:43 on dist-test-slave-s11t"
server_key: "521f03e8f2e2f79c903c1c7617386116"
server_key_iv: "2c9d4cb8a63c869f68a60639fdf7fb1a"
server_key_version: "encryptionkey@0"
I20260809 02:15:43.343094 15698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/minicluster-data/master-0/wal/instance:
uuid: "f52e52b03ac6427889a60294bd825a1b"
format_stamp: "Formatted at 2026-08-09 02:15:43 on dist-test-slave-s11t"
server_key: "521f03e8f2e2f79c903c1c7617386116"
server_key_iv: "2c9d4cb8a63c869f68a60639fdf7fb1a"
server_key_version: "encryptionkey@0"
I20260809 02:15:43.346872 15698 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 02:15:43.349370 15719 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:43.350589 15698 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 02:15:43.350767 15698 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/minicluster-data/master-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/minicluster-data/master-0/wal
uuid: "f52e52b03ac6427889a60294bd825a1b"
format_stamp: "Formatted at 2026-08-09 02:15:43 on dist-test-slave-s11t"
server_key: "521f03e8f2e2f79c903c1c7617386116"
server_key_iv: "2c9d4cb8a63c869f68a60639fdf7fb1a"
server_key_version: "encryptionkey@0"
I20260809 02:15:43.350890 15698 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:15:43.365061 15698 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:15:43.368047 15698 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:15:43.368255 15698 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:15:43.375841 15698 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.190:40021
I20260809 02:15:43.375859 15799 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.190:40021 every 8 connection(s)
I20260809 02:15:43.376879 15698 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
I20260809 02:15:43.378510 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 15698
I20260809 02:15:43.378664 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/minicluster-data/master-0/wal/instance
I20260809 02:15:43.379707 15800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:15:43.379993 15662 external_mini_cluster.cc:1428] Setting key 783529c2d8c8ddb6ba16365c3d124b3c
I20260809 02:15:43.386178 15800 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f52e52b03ac6427889a60294bd825a1b: Bootstrap starting.
I20260809 02:15:43.388335 15800 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f52e52b03ac6427889a60294bd825a1b: Neither blocks nor log segments found. Creating new log.
I20260809 02:15:43.389664 15800 log.cc:824] T 00000000000000000000000000000000 P f52e52b03ac6427889a60294bd825a1b: Log is configured to *not* fsync() on all Append() calls
I20260809 02:15:43.391669 15800 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f52e52b03ac6427889a60294bd825a1b: No bootstrap required, opened a new log
I20260809 02:15:43.396636 15800 raft_consensus.cc:348] T 00000000000000000000000000000000 P f52e52b03ac6427889a60294bd825a1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f52e52b03ac6427889a60294bd825a1b" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 40021 } }
I20260809 02:15:43.396891 15800 raft_consensus.cc:374] T 00000000000000000000000000000000 P f52e52b03ac6427889a60294bd825a1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:15:43.396986 15800 raft_consensus.cc:729] T 00000000000000000000000000000000 P f52e52b03ac6427889a60294bd825a1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f52e52b03ac6427889a60294bd825a1b, State: Initialized, Role: FOLLOWER
I20260809 02:15:43.397528 15800 consensus_queue.cc:260] T 00000000000000000000000000000000 P f52e52b03ac6427889a60294bd825a1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f52e52b03ac6427889a60294bd825a1b" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 40021 } }
I20260809 02:15:43.397701 15800 raft_consensus.cc:388] T 00000000000000000000000000000000 P f52e52b03ac6427889a60294bd825a1b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:15:43.397766 15800 raft_consensus.cc:482] T 00000000000000000000000000000000 P f52e52b03ac6427889a60294bd825a1b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
Aug 09 02:15:43 dist-test-slave-s11t krb5kdc[15672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241743, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:15:43.397867 15800 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f52e52b03ac6427889a60294bd825a1b [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:15:43.398890 15800 raft_consensus.cc:504] T 00000000000000000000000000000000 P f52e52b03ac6427889a60294bd825a1b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f52e52b03ac6427889a60294bd825a1b" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 40021 } }
I20260809 02:15:43.399269 15800 leader_election.cc:302] T 00000000000000000000000000000000 P f52e52b03ac6427889a60294bd825a1b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f52e52b03ac6427889a60294bd825a1b; no voters:
I20260809 02:15:43.399622 15800 leader_election.cc:288] T 00000000000000000000000000000000 P f52e52b03ac6427889a60294bd825a1b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:15:43.399717 15807 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f52e52b03ac6427889a60294bd825a1b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:15:43.400843 15807 raft_consensus.cc:686] T 00000000000000000000000000000000 P f52e52b03ac6427889a60294bd825a1b [term 1 LEADER]: Becoming Leader. State: Replica: f52e52b03ac6427889a60294bd825a1b, State: Running, Role: LEADER
I20260809 02:15:43.400888 15800 sys_catalog.cc:564] T 00000000000000000000000000000000 P f52e52b03ac6427889a60294bd825a1b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:15:43.401166 15807 consensus_queue.cc:237] T 00000000000000000000000000000000 P f52e52b03ac6427889a60294bd825a1b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f52e52b03ac6427889a60294bd825a1b" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 40021 } }
I20260809 02:15:43.404917 15808 sys_catalog.cc:455] T 00000000000000000000000000000000 P f52e52b03ac6427889a60294bd825a1b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f52e52b03ac6427889a60294bd825a1b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f52e52b03ac6427889a60294bd825a1b" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 40021 } } }
I20260809 02:15:43.405411 15808 sys_catalog.cc:458] T 00000000000000000000000000000000 P f52e52b03ac6427889a60294bd825a1b [sys.catalog]: This master's current role is: LEADER
I20260809 02:15:43.405303 15810 sys_catalog.cc:455] T 00000000000000000000000000000000 P f52e52b03ac6427889a60294bd825a1b [sys.catalog]: SysCatalogTable state changed. Reason: New leader f52e52b03ac6427889a60294bd825a1b. Latest consensus state: current_term: 1 leader_uuid: "f52e52b03ac6427889a60294bd825a1b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f52e52b03ac6427889a60294bd825a1b" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 40021 } } }
I20260809 02:15:43.405550 15810 sys_catalog.cc:458] T 00000000000000000000000000000000 P f52e52b03ac6427889a60294bd825a1b [sys.catalog]: This master's current role is: LEADER
I20260809 02:15:43.405589 15804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:43.383226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55784 (local address 127.15.75.190:40021)
0809 02:15:43.383683 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:43.383699 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:43.384809 (+ 1110us) server_negotiation.cc:407] Connection header received
0809 02:15:43.387886 (+ 3077us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:43.387902 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:43.388189 (+ 287us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:43.388518 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:43.390907 (+ 2389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:43.391919 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:43.393327 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:43.393785 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:43.393973 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:43.398689 (+ 4716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:15:43.398694 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:15:43.398705 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:15:43.398738 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:15:43.400605 (+ 1867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:43.401444 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:43.401449 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:43.401453 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:43.401565 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:43.401964 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:43.401972 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:43.401973 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:43.402540 (+ 567us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:15:43.402765 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:43.403741 (+ 976us) server_negotiation.cc:299] Negotiation successful
0809 02:15:43.404039 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":138,"threads_started":1}
I20260809 02:15:43.406104 15827 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:15:43.409574 15827 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:15:43.415690 15827 catalog_manager.cc:1347] Generated new cluster ID: f6242614a3824f2fa32d40dee6d58fd3
I20260809 02:15:43.415769 15827 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:15:43.428836 15827 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:15:43.430311 15827 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:15:43.440119 15827 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f52e52b03ac6427889a60294bd825a1b: Generated new TSK 0
I20260809 02:15:43.441138 15827 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.75.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab.
Entry for principal kudu/127.15.75.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab.
I20260809 02:15:43.473218 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.129:0
--local_ip_for_outbound_sockets=127.15.75.129
--webserver_interface=127.15.75.129
--webserver_port=0
--tserver_master_addrs=127.15.75.190:40021
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab
--principal=kudu/127.15.75.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:36643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:15:43.621275 15832 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:15:43.621501 15832 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:15:43.621542 15832 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:15:43.625447 15832 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:15:43.625496 15832 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:15:43.625552 15832 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.129
I20260809 02:15:43.630491 15832 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:36643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab
--principal=kudu/127.15.75.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.75.129
--webserver_port=0
--tserver_master_addrs=127.15.75.190:40021
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15832
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.129
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:15:43.631394 15832 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:15:43.632299 15832 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:15:43.641623 15839 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:15:43.641757 15838 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:15:43.641640 15841 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:15:43.642099 15832 server_base.cc:1034] running on GCE node
I20260809 02:15:43.642511 15832 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:15:43.643090 15832 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:15:43.644244 15832 hybrid_clock.cc:648] HybridClock initialized: now 1786241743644209 us; error 47 us; skew 500 ppm
Aug 09 02:15:43 dist-test-slave-s11t krb5kdc[15672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241743, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:15:43.647092 15832 init.cc:377] Logged in from keytab as kudu/127.15.75.129@KRBTEST.COM (short username kudu)
I20260809 02:15:43.647950 15832 webserver.cc:460] Webserver started at http://127.15.75.129:39503/ using document root <none> and password file <none>
I20260809 02:15:43.648456 15832 fs_manager.cc:359] Metadata directory not provided
I20260809 02:15:43.648514 15832 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:15:43.648675 15832 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:15:43.650681 15832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/minicluster-data/ts-0/data/instance:
uuid: "de97b80ae0e8478ba44e2411a4dd3303"
format_stamp: "Formatted at 2026-08-09 02:15:43 on dist-test-slave-s11t"
server_key: "b57146222665592881f663e84d1d8492"
server_key_iv: "0671e71b8bad711086762b0ce03ca1e6"
server_key_version: "encryptionkey@0"
I20260809 02:15:43.651175 15832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/minicluster-data/ts-0/wal/instance:
uuid: "de97b80ae0e8478ba44e2411a4dd3303"
format_stamp: "Formatted at 2026-08-09 02:15:43 on dist-test-slave-s11t"
server_key: "b57146222665592881f663e84d1d8492"
server_key_iv: "0671e71b8bad711086762b0ce03ca1e6"
server_key_version: "encryptionkey@0"
I20260809 02:15:43.654971 15832 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260809 02:15:43.657541 15853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:43.658710 15832 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260809 02:15:43.658859 15832 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/minicluster-data/ts-0/wal
uuid: "de97b80ae0e8478ba44e2411a4dd3303"
format_stamp: "Formatted at 2026-08-09 02:15:43 on dist-test-slave-s11t"
server_key: "b57146222665592881f663e84d1d8492"
server_key_iv: "0671e71b8bad711086762b0ce03ca1e6"
server_key_version: "encryptionkey@0"
I20260809 02:15:43.658972 15832 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:15:43.680001 15832 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:15:43.683465 15832 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:15:43.683665 15832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:15:43.684259 15832 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:15:43.685094 15832 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:15:43.685166 15832 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:43.685240 15832 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:15:43.685283 15832 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:43.693842 15832 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.129:38421
I20260809 02:15:43.693917 16018 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.129:38421 every 8 connection(s)
I20260809 02:15:43.694729 15832 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
I20260809 02:15:43.700754 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 15832
I20260809 02:15:43.700866 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/minicluster-data/ts-0/wal/instance
I20260809 02:15:43.701139 15662 external_mini_cluster.cc:1428] Setting key 9f5b6c080c4f7302abdc49c26737aeb8
Aug 09 02:15:43 dist-test-slave-s11t krb5kdc[15672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241743, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.129@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:15:43.709424 15804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:43.696593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.129:38441 (local address 127.15.75.190:40021)
0809 02:15:43.696752 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:43.696756 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:43.697763 (+ 1007us) server_negotiation.cc:407] Connection header received
0809 02:15:43.698544 (+ 781us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:43.698547 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:43.698593 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:43.698666 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:43.699717 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:43.700254 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:43.702447 (+ 2193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:43.702765 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:43.702886 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:43.705585 (+ 2699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:15:43.705591 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:15:43.705595 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:15:43.705628 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:15:43.707238 (+ 1610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:43.707806 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:43.707809 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:43.707811 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:43.707899 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:43.708260 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:43.708263 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:43.708265 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:43.708452 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:15:43.708564 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:43.709120 (+ 556us) server_negotiation.cc:299] Negotiation successful
0809 02:15:43.709252 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 02:15:43.710170 16022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:43.696998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:40021 (local address 127.15.75.129:38441)
0809 02:15:43.697612 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:43.697650 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:43.698374 (+ 724us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:43.698803 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:43.698809 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:43.699154 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:43.699594 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:43.699606 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:43.700422 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:43.700425 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:43.701861 (+ 1436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:43.701873 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:43.702924 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:43.702927 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:43.703066 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:43.703647 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:43.703663 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:43.705413 (+ 1750us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:15:43.707403 (+ 1990us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:15:43.707409 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:15:43.707423 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:15:43.707684 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:15:43.708004 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:15:43.708007 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:15:43.708009 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:15:43.708149 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:15:43.708565 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:15:43.708572 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:15:43.708864 (+ 292us) client_negotiation.cc:769] Sending connection context
0809 02:15:43.709054 (+ 190us) client_negotiation.cc:240] Negotiation successful
0809 02:15:43.709380 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":108,"threads_started":1}
I20260809 02:15:43.711457 16019 heartbeater.cc:347] Connected to a master server at 127.15.75.190:40021
I20260809 02:15:43.711696 16019 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.15.75.130@KRBTEST.COM; defaulting to no policy
I20260809 02:15:43.712208 16019 heartbeater.cc:511] Master 127.15.75.190:40021 requested a full tablet report, sending...
I20260809 02:15:43.713754 15745 ts_manager.cc:194] Registered new tserver with Master: de97b80ae0e8478ba44e2411a4dd3303 (127.15.75.129:38421)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.130@KRBTEST.COM" created.
I20260809 02:15:43.714854 15745 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.75.129@KRBTEST.COM'} at 127.15.75.129:38441
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab.
Entry for principal kudu/127.15.75.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab.
I20260809 02:15:43.731097 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.130:0
--local_ip_for_outbound_sockets=127.15.75.130
--webserver_interface=127.15.75.130
--webserver_port=0
--tserver_master_addrs=127.15.75.190:40021
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab
--principal=kudu/127.15.75.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:36643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:15:43.842494 16027 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:15:43.842722 16027 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:15:43.842765 16027 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:15:43.846692 16027 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:15:43.846738 16027 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:15:43.846793 16027 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.130
I20260809 02:15:43.851763 16027 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:36643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab
--principal=kudu/127.15.75.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.75.130
--webserver_port=0
--tserver_master_addrs=127.15.75.190:40021
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16027
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.130
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:15:43.852622 16027 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:15:43.853585 16027 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:15:43.863401 16040 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:15:43.863409 16038 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:15:43.863538 16037 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:15:43.863740 16027 server_base.cc:1034] running on GCE node
I20260809 02:15:43.864189 16027 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:15:43.864861 16027 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:15:43.866068 16027 hybrid_clock.cc:648] HybridClock initialized: now 1786241743866059 us; error 30 us; skew 500 ppm
Aug 09 02:15:43 dist-test-slave-s11t krb5kdc[15672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241743, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:15:43.868788 16027 init.cc:377] Logged in from keytab as kudu/127.15.75.130@KRBTEST.COM (short username kudu)
I20260809 02:15:43.869565 16027 webserver.cc:460] Webserver started at http://127.15.75.130:33409/ using document root <none> and password file <none>
I20260809 02:15:43.870097 16027 fs_manager.cc:359] Metadata directory not provided
I20260809 02:15:43.870157 16027 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:15:43.870317 16027 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:15:43.872305 16027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/minicluster-data/ts-1/data/instance:
uuid: "fde2d60d3f774d3c81f8532ff5f556e0"
format_stamp: "Formatted at 2026-08-09 02:15:43 on dist-test-slave-s11t"
server_key: "6000db83712204aa10a76b14d333a0ec"
server_key_iv: "105730e9305b40aa01d8cc2af51669d5"
server_key_version: "encryptionkey@0"
I20260809 02:15:43.872927 16027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/minicluster-data/ts-1/wal/instance:
uuid: "fde2d60d3f774d3c81f8532ff5f556e0"
format_stamp: "Formatted at 2026-08-09 02:15:43 on dist-test-slave-s11t"
server_key: "6000db83712204aa10a76b14d333a0ec"
server_key_iv: "105730e9305b40aa01d8cc2af51669d5"
server_key_version: "encryptionkey@0"
I20260809 02:15:43.876770 16027 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 02:15:43.879092 16052 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:43.880267 16027 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 02:15:43.880388 16027 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/minicluster-data/ts-1/wal
uuid: "fde2d60d3f774d3c81f8532ff5f556e0"
format_stamp: "Formatted at 2026-08-09 02:15:43 on dist-test-slave-s11t"
server_key: "6000db83712204aa10a76b14d333a0ec"
server_key_iv: "105730e9305b40aa01d8cc2af51669d5"
server_key_version: "encryptionkey@0"
I20260809 02:15:43.880491 16027 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:15:43.905443 16027 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:15:43.908677 16027 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:15:43.908932 16027 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:15:43.909509 16027 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:15:43.910318 16027 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:15:43.910389 16027 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:43.910461 16027 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:15:43.910514 16027 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:43.919320 16027 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.130:41629
I20260809 02:15:43.919355 16213 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.130:41629 every 8 connection(s)
I20260809 02:15:43.920159 16027 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
I20260809 02:15:43.927374 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 16027
I20260809 02:15:43.927470 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/minicluster-data/ts-1/wal/instance
I20260809 02:15:43.927708 15662 external_mini_cluster.cc:1428] Setting key 4a2af1a95b082e803a8d413ef9198ac6
Aug 09 02:15:43 dist-test-slave-s11t krb5kdc[15672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241743, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.130@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:15:43.935235 15804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:43.922151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.130:45695 (local address 127.15.75.190:40021)
0809 02:15:43.922297 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:43.922300 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:43.923385 (+ 1085us) server_negotiation.cc:407] Connection header received
0809 02:15:43.924371 (+ 986us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:43.924375 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:43.924461 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:43.924565 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:43.926034 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:43.926560 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:43.927814 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:43.928043 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:43.928161 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:43.930920 (+ 2759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:15:43.930924 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:15:43.930928 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:15:43.930956 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:15:43.933202 (+ 2246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:43.933717 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:43.933721 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:43.933723 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:43.933806 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:43.934152 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:43.934157 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:43.934158 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:43.934346 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:15:43.934457 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:43.934944 (+ 487us) server_negotiation.cc:299] Negotiation successful
0809 02:15:43.935067 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 02:15:43.936033 16216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:43.922506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:40021 (local address 127.15.75.130:45695)
0809 02:15:43.923243 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:43.923284 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:43.924129 (+ 845us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:43.924783 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:43.924794 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:43.925340 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:43.925890 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:43.925903 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:43.926685 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:43.926688 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:43.927671 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:43.927681 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:43.928201 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:43.928206 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:43.928417 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:43.929063 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:43.929083 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:43.930751 (+ 1668us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:15:43.933341 (+ 2590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:15:43.933347 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:15:43.933361 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:15:43.933611 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:15:43.933910 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:15:43.933915 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:15:43.933917 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:15:43.934054 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:15:43.934457 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:15:43.934464 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:15:43.934699 (+ 235us) client_negotiation.cc:769] Sending connection context
0809 02:15:43.934887 (+ 188us) client_negotiation.cc:240] Negotiation successful
0809 02:15:43.935197 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":118,"threads_started":1}
I20260809 02:15:43.937155 16214 heartbeater.cc:347] Connected to a master server at 127.15.75.190:40021
I20260809 02:15:43.937417 16214 heartbeater.cc:464] Registering TS with master...
I20260809 02:15:43.937907 16214 heartbeater.cc:511] Master 127.15.75.190:40021 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.15.75.131@KRBTEST.COM; defaulting to no policy
I20260809 02:15:43.938921 15745 ts_manager.cc:194] Registered new tserver with Master: fde2d60d3f774d3c81f8532ff5f556e0 (127.15.75.130:41629)
I20260809 02:15:43.939585 15745 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.75.130@KRBTEST.COM'} at 127.15.75.130:45695
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab.
Entry for principal kudu/127.15.75.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab.
I20260809 02:15:43.955636 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.131:0
--local_ip_for_outbound_sockets=127.15.75.131
--webserver_interface=127.15.75.131
--webserver_port=0
--tserver_master_addrs=127.15.75.190:40021
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab
--principal=kudu/127.15.75.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:36643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:15:44.069933 16220 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:15:44.070169 16220 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:15:44.070211 16220 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:15:44.074175 16220 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:15:44.074231 16220 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:15:44.074287 16220 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.131
I20260809 02:15:44.079252 16220 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:36643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab
--principal=kudu/127.15.75.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.75.131
--webserver_port=0
--tserver_master_addrs=127.15.75.190:40021
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16220
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.131
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:15:44.080154 16220 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:15:44.081151 16220 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:15:44.090373 16230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:15:44.090387 16227 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:15:44.090454 16226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:15:44.090636 16220 server_base.cc:1034] running on GCE node
I20260809 02:15:44.091090 16220 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:15:44.091667 16220 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:15:44.092861 16220 hybrid_clock.cc:648] HybridClock initialized: now 1786241744092849 us; error 24 us; skew 500 ppm
Aug 09 02:15:44 dist-test-slave-s11t krb5kdc[15672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241744, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:15:44.095513 16220 init.cc:377] Logged in from keytab as kudu/127.15.75.131@KRBTEST.COM (short username kudu)
I20260809 02:15:44.096313 16220 webserver.cc:460] Webserver started at http://127.15.75.131:39259/ using document root <none> and password file <none>
I20260809 02:15:44.096917 16220 fs_manager.cc:359] Metadata directory not provided
I20260809 02:15:44.097008 16220 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:15:44.097231 16220 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:15:44.099192 16220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/minicluster-data/ts-2/data/instance:
uuid: "4659e8e12c9f4962bec03d5c0701a502"
format_stamp: "Formatted at 2026-08-09 02:15:44 on dist-test-slave-s11t"
server_key: "531fc1d3549e11713cf5948c3416877e"
server_key_iv: "db1947393a4d88da040fb83f0b3e66d2"
server_key_version: "encryptionkey@0"
I20260809 02:15:44.099786 16220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/minicluster-data/ts-2/wal/instance:
uuid: "4659e8e12c9f4962bec03d5c0701a502"
format_stamp: "Formatted at 2026-08-09 02:15:44 on dist-test-slave-s11t"
server_key: "531fc1d3549e11713cf5948c3416877e"
server_key_iv: "db1947393a4d88da040fb83f0b3e66d2"
server_key_version: "encryptionkey@0"
I20260809 02:15:44.103788 16220 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260809 02:15:44.106096 16244 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:44.107259 16220 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 02:15:44.107403 16220 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/minicluster-data/ts-2/wal
uuid: "4659e8e12c9f4962bec03d5c0701a502"
format_stamp: "Formatted at 2026-08-09 02:15:44 on dist-test-slave-s11t"
server_key: "531fc1d3549e11713cf5948c3416877e"
server_key_iv: "db1947393a4d88da040fb83f0b3e66d2"
server_key_version: "encryptionkey@0"
I20260809 02:15:44.107513 16220 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:15:44.122997 16220 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:15:44.125758 16220 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:15:44.125955 16220 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:15:44.126507 16220 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:15:44.127296 16220 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:15:44.127367 16220 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:44.127432 16220 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:15:44.127475 16220 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:44.139065 16220 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.131:42367
I20260809 02:15:44.139099 16415 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.131:42367 every 8 connection(s)
I20260809 02:15:44.139927 16220 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
I20260809 02:15:44.141796 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 16220
I20260809 02:15:44.141930 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241741788589-15662-0/minicluster-data/ts-2/wal/instance
I20260809 02:15:44.142189 15662 external_mini_cluster.cc:1428] Setting key 7935ebf97eb43b5b16dfbea61e3cad54
Aug 09 02:15:44 dist-test-slave-s11t krb5kdc[15672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241744, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.131@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:15:44.152842 15804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:44.141823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.131:43351 (local address 127.15.75.190:40021)
0809 02:15:44.142781 (+ 958us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:44.142788 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:44.143157 (+ 369us) server_negotiation.cc:407] Connection header received
0809 02:15:44.144053 (+ 896us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:44.144055 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:44.144100 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:44.144166 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:44.145394 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:44.145917 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:44.147003 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:44.147210 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:44.147283 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:44.149648 (+ 2365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:15:44.149651 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:15:44.149653 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:15:44.149676 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:15:44.151023 (+ 1347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:44.151515 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:44.151517 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:44.151519 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:44.151591 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:44.151887 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:44.151889 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:44.151891 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:44.152054 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:15:44.152127 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:44.152561 (+ 434us) server_negotiation.cc:299] Negotiation successful
0809 02:15:44.152665 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":838}
I20260809 02:15:44.153599 16418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:44.142215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:40021 (local address 127.15.75.131:43351)
0809 02:15:44.143023 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:44.143074 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:44.143849 (+ 775us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:44.144338 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:44.144346 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:44.144805 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:44.145260 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:44.145273 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:44.146025 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:44.146027 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:44.146902 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:44.146911 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:44.147330 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:44.147333 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:44.147475 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:44.147994 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:44.148009 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:44.149503 (+ 1494us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:15:44.151132 (+ 1629us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:15:44.151137 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:15:44.151147 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:15:44.151427 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:15:44.151671 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:15:44.151673 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:15:44.151675 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:15:44.151808 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:15:44.152143 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:15:44.152147 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:15:44.152362 (+ 215us) client_negotiation.cc:769] Sending connection context
0809 02:15:44.152512 (+ 150us) client_negotiation.cc:240] Negotiation successful
0809 02:15:44.152830 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":603,"thread_start_us":137,"threads_started":1}
I20260809 02:15:44.154721 16416 heartbeater.cc:347] Connected to a master server at 127.15.75.190:40021
I20260809 02:15:44.154940 16416 heartbeater.cc:464] Registering TS with master...
I20260809 02:15:44.155507 16416 heartbeater.cc:511] Master 127.15.75.190:40021 requested a full tablet report, sending...
I20260809 02:15:44.156524 15744 ts_manager.cc:194] Registered new tserver with Master: 4659e8e12c9f4962bec03d5c0701a502 (127.15.75.131:42367)
I20260809 02:15:44.157132 15744 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.75.131@KRBTEST.COM'} at 127.15.75.131:43351
I20260809 02:15:44.168095 15662 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 02:15:44 dist-test-slave-s11t krb5kdc[15672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241744, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 02:15:44 dist-test-slave-s11t krb5kdc[15672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241744, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.75.129@KRBTEST.COM
I20260809 02:15:44.195420 16428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:44.186081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56978 (local address 127.15.75.129:38421)
0809 02:15:44.186384 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:44.186400 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:44.186425 (+ 25us) server_negotiation.cc:407] Connection header received
0809 02:15:44.186477 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:44.186481 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:44.186660 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:44.186796 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:44.187451 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:44.188240 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:44.189115 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:44.189484 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:44.189570 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:44.191981 (+ 2411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:15:44.191990 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:15:44.192003 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:15:44.192026 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:15:44.193629 (+ 1603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:44.194067 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:44.194073 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:44.194074 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:44.194176 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:44.194440 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:44.194442 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:44.194443 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:44.194671 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:15:44.194834 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:44.195054 (+ 220us) server_negotiation.cc:299] Negotiation successful
0809 02:15:44.195251 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":90,"threads_started":1}
W20260809 02:15:44.196545 15895 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56978
Aug 09 02:15:44 dist-test-slave-s11t krb5kdc[15672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241744, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 02:15:44 dist-test-slave-s11t krb5kdc[15672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241744, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.75.129@KRBTEST.COM
I20260809 02:15:44.222483 16428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:44.214614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56988 (local address 127.15.75.129:38421)
0809 02:15:44.214819 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:44.214823 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:44.214843 (+ 20us) server_negotiation.cc:407] Connection header received
0809 02:15:44.214960 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:44.214963 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:44.215002 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:44.215070 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:44.215687 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:44.216202 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:44.217040 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:44.217271 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:44.217352 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:44.219569 (+ 2217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:15:44.219572 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:15:44.219574 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:15:44.219596 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:15:44.220903 (+ 1307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:44.221292 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:44.221296 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:44.221297 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:44.221380 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:44.221633 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:44.221635 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:44.221637 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:44.221819 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:15:44.221921 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:44.222173 (+ 252us) server_negotiation.cc:299] Negotiation successful
0809 02:15:44.222350 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260809 02:15:44.223565 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 15832
I20260809 02:15:44.238087 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 16027
I20260809 02:15:44.252822 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 16220
I20260809 02:15:44.284324 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 15698
2026-08-09T02:15:44Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2493 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:15:44 dist-test-slave-s11t krb5kdc[16438](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:15:44 dist-test-slave-s11t krb5kdc[16438](info): set up 2 sockets
Aug 09 02:15:44 dist-test-slave-s11t krb5kdc[16438](info): commencing operation
krb5kdc: starting...
W20260809 02:15:45.529212 15662 mini_kdc.cc:121] Time spent starting KDC: real 1.217s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Aug 09 02:15:45 dist-test-slave-s11t krb5kdc[16438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241745, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:15:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:15:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.75.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab.
Entry for principal kudu/127.15.75.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab.
I20260809 02:15:45.649361 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:36703
--webserver_interface=127.15.75.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab
--principal=kudu/127.15.75.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:36655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.75.190:36703
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:15:45.760185 16457 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:15:45.760466 16457 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:15:45.760545 16457 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:15:45.764569 16457 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:15:45.764655 16457 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:15:45.764695 16457 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:15:45.764779 16457 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:15:45.764816 16457 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:15:45.769882 16457 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:36655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.75.190:36703
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab
--principal=kudu/127.15.75.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:36703
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.75.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16457
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:15:45.770897 16457 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:15:45.771867 16457 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:15:45.780165 16471 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:15:45.780202 16464 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:15:45.780202 16467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:15:45.780264 16457 server_base.cc:1034] running on GCE node
I20260809 02:15:45.781093 16457 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:15:45.782117 16457 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:15:45.783308 16457 hybrid_clock.cc:648] HybridClock initialized: now 1786241745783294 us; error 36 us; skew 500 ppm
Aug 09 02:15:45 dist-test-slave-s11t krb5kdc[16438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241745, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:15:45.790356 16457 init.cc:377] Logged in from keytab as kudu/127.15.75.190@KRBTEST.COM (short username kudu)
I20260809 02:15:45.791193 16457 webserver.cc:460] Webserver started at http://127.15.75.190:37513/ using document root <none> and password file <none>
I20260809 02:15:45.791695 16457 fs_manager.cc:359] Metadata directory not provided
I20260809 02:15:45.791751 16457 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:15:45.791915 16457 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:15:45.793917 16457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/minicluster-data/master-0/data/instance:
uuid: "a95fe48ad008464e9282dcfa62108112"
format_stamp: "Formatted at 2026-08-09 02:15:45 on dist-test-slave-s11t"
server_key: "28c78095a04c394ed5a27dd183abfebf"
server_key_iv: "003385bc0a0137d5164e00c3e9ef3837"
server_key_version: "encryptionkey@0"
I20260809 02:15:45.794423 16457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/minicluster-data/master-0/wal/instance:
uuid: "a95fe48ad008464e9282dcfa62108112"
format_stamp: "Formatted at 2026-08-09 02:15:45 on dist-test-slave-s11t"
server_key: "28c78095a04c394ed5a27dd183abfebf"
server_key_iv: "003385bc0a0137d5164e00c3e9ef3837"
server_key_version: "encryptionkey@0"
I20260809 02:15:45.798055 16457 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 02:15:45.800441 16481 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:45.801766 16457 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 02:15:45.801908 16457 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/minicluster-data/master-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/minicluster-data/master-0/wal
uuid: "a95fe48ad008464e9282dcfa62108112"
format_stamp: "Formatted at 2026-08-09 02:15:45 on dist-test-slave-s11t"
server_key: "28c78095a04c394ed5a27dd183abfebf"
server_key_iv: "003385bc0a0137d5164e00c3e9ef3837"
server_key_version: "encryptionkey@0"
I20260809 02:15:45.802021 16457 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:15:45.824402 16457 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:15:45.827973 16457 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:15:45.828190 16457 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:15:45.836323 16457 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.190:36703
I20260809 02:15:45.836272 16568 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.190:36703 every 8 connection(s)
I20260809 02:15:45.837760 16457 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
I20260809 02:15:45.840492 16571 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:15:45.845395 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 16457
I20260809 02:15:45.845490 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/minicluster-data/master-0/wal/instance
I20260809 02:15:45.845758 15662 external_mini_cluster.cc:1428] Setting key 02edaabf8a661364ff8857fba981d495
I20260809 02:15:45.846275 16571 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a95fe48ad008464e9282dcfa62108112: Bootstrap starting.
I20260809 02:15:45.848793 16571 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a95fe48ad008464e9282dcfa62108112: Neither blocks nor log segments found. Creating new log.
I20260809 02:15:45.849781 16571 log.cc:824] T 00000000000000000000000000000000 P a95fe48ad008464e9282dcfa62108112: Log is configured to *not* fsync() on all Append() calls
I20260809 02:15:45.851894 16571 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a95fe48ad008464e9282dcfa62108112: No bootstrap required, opened a new log
Aug 09 02:15:45 dist-test-slave-s11t krb5kdc[16438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241745, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:15:45.856282 16571 raft_consensus.cc:348] T 00000000000000000000000000000000 P a95fe48ad008464e9282dcfa62108112 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a95fe48ad008464e9282dcfa62108112" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 36703 } }
I20260809 02:15:45.856551 16571 raft_consensus.cc:374] T 00000000000000000000000000000000 P a95fe48ad008464e9282dcfa62108112 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:15:45.856619 16571 raft_consensus.cc:729] T 00000000000000000000000000000000 P a95fe48ad008464e9282dcfa62108112 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a95fe48ad008464e9282dcfa62108112, State: Initialized, Role: FOLLOWER
I20260809 02:15:45.857059 16571 consensus_queue.cc:260] T 00000000000000000000000000000000 P a95fe48ad008464e9282dcfa62108112 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a95fe48ad008464e9282dcfa62108112" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 36703 } }
I20260809 02:15:45.857249 16571 raft_consensus.cc:388] T 00000000000000000000000000000000 P a95fe48ad008464e9282dcfa62108112 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:15:45.857327 16571 raft_consensus.cc:482] T 00000000000000000000000000000000 P a95fe48ad008464e9282dcfa62108112 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:15:45.857460 16571 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a95fe48ad008464e9282dcfa62108112 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:15:45.861344 16577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:45.847173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44364 (local address 127.15.75.190:36703)
0809 02:15:45.847631 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:45.847645 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:45.847694 (+ 49us) server_negotiation.cc:407] Connection header received
0809 02:15:45.848327 (+ 633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:45.848343 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:45.848680 (+ 337us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:45.849064 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:45.849993 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:45.851268 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:45.852435 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:45.852994 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:45.853111 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:45.856151 (+ 3040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:15:45.856156 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:15:45.856170 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:15:45.856196 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:15:45.858228 (+ 2032us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:45.858692 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:45.858697 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:45.858703 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:45.858863 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:45.859140 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:45.859146 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:45.859150 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:45.859537 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:15:45.859748 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:45.860092 (+ 344us) server_negotiation.cc:299] Negotiation successful
0809 02:15:45.860387 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":141,"threads_started":1}
I20260809 02:15:45.870814 16571 raft_consensus.cc:504] T 00000000000000000000000000000000 P a95fe48ad008464e9282dcfa62108112 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a95fe48ad008464e9282dcfa62108112" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 36703 } }
I20260809 02:15:45.871209 16571 leader_election.cc:302] T 00000000000000000000000000000000 P a95fe48ad008464e9282dcfa62108112 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a95fe48ad008464e9282dcfa62108112; no voters:
I20260809 02:15:45.871549 16571 leader_election.cc:288] T 00000000000000000000000000000000 P a95fe48ad008464e9282dcfa62108112 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:15:45.871685 16579 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a95fe48ad008464e9282dcfa62108112 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:15:45.871914 16579 raft_consensus.cc:686] T 00000000000000000000000000000000 P a95fe48ad008464e9282dcfa62108112 [term 1 LEADER]: Becoming Leader. State: Replica: a95fe48ad008464e9282dcfa62108112, State: Running, Role: LEADER
I20260809 02:15:45.872247 16579 consensus_queue.cc:237] T 00000000000000000000000000000000 P a95fe48ad008464e9282dcfa62108112 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a95fe48ad008464e9282dcfa62108112" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 36703 } }
I20260809 02:15:45.872885 16571 sys_catalog.cc:564] T 00000000000000000000000000000000 P a95fe48ad008464e9282dcfa62108112 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:15:45.874627 16580 sys_catalog.cc:455] T 00000000000000000000000000000000 P a95fe48ad008464e9282dcfa62108112 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a95fe48ad008464e9282dcfa62108112" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a95fe48ad008464e9282dcfa62108112" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 36703 } } }
I20260809 02:15:45.875326 16580 sys_catalog.cc:458] T 00000000000000000000000000000000 P a95fe48ad008464e9282dcfa62108112 [sys.catalog]: This master's current role is: LEADER
I20260809 02:15:45.874366 16582 sys_catalog.cc:455] T 00000000000000000000000000000000 P a95fe48ad008464e9282dcfa62108112 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a95fe48ad008464e9282dcfa62108112. Latest consensus state: current_term: 1 leader_uuid: "a95fe48ad008464e9282dcfa62108112" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a95fe48ad008464e9282dcfa62108112" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 36703 } } }
I20260809 02:15:45.875735 16582 sys_catalog.cc:458] T 00000000000000000000000000000000 P a95fe48ad008464e9282dcfa62108112 [sys.catalog]: This master's current role is: LEADER
I20260809 02:15:45.876166 16597 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:15:45.879130 16597 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:15:45.884447 16597 catalog_manager.cc:1347] Generated new cluster ID: 8a989880c59f4a41886b15e39dd96247
I20260809 02:15:45.884507 16597 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:15:45.899124 16597 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:15:45.900396 16597 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:15:45.907217 16597 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a95fe48ad008464e9282dcfa62108112: Generated new TSK 0
I20260809 02:15:45.907992 16597 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.75.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab.
Entry for principal kudu/127.15.75.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab.
W20260809 02:15:46.624107 15662 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.15.75.129: real 0.712s user 0.000s sys 0.005s
I20260809 02:15:46.624863 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.129:0
--local_ip_for_outbound_sockets=127.15.75.129
--webserver_interface=127.15.75.129
--webserver_port=0
--tserver_master_addrs=127.15.75.190:36703
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab
--principal=kudu/127.15.75.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:36655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:15:46.742436 16616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:15:46.742725 16616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:15:46.742799 16616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:15:46.746896 16616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:15:46.746974 16616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:15:46.747061 16616 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.129
I20260809 02:15:46.752348 16616 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:36655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab
--principal=kudu/127.15.75.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.75.129
--webserver_port=0
--tserver_master_addrs=127.15.75.190:36703
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16616
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.129
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:15:46.753409 16616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:15:46.754468 16616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:15:46.764389 16628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:15:46.764389 16624 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:15:46.764393 16625 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:15:46.764475 16616 server_base.cc:1034] running on GCE node
I20260809 02:15:46.765048 16616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:15:46.765647 16616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:15:46.766808 16616 hybrid_clock.cc:648] HybridClock initialized: now 1786241746766774 us; error 44 us; skew 500 ppm
Aug 09 02:15:46 dist-test-slave-s11t krb5kdc[16438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241746, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:15:46.769639 16616 init.cc:377] Logged in from keytab as kudu/127.15.75.129@KRBTEST.COM (short username kudu)
I20260809 02:15:46.770507 16616 webserver.cc:460] Webserver started at http://127.15.75.129:34533/ using document root <none> and password file <none>
I20260809 02:15:46.771001 16616 fs_manager.cc:359] Metadata directory not provided
I20260809 02:15:46.771059 16616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:15:46.771289 16616 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:15:46.773880 16616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/minicluster-data/ts-0/data/instance:
uuid: "ecef09984f904f65bdd54323d956122f"
format_stamp: "Formatted at 2026-08-09 02:15:46 on dist-test-slave-s11t"
server_key: "ec9ceb0821513b20d8bfa54cfbc4957d"
server_key_iv: "f1543de3e48b695fd5b9baa4141a71cb"
server_key_version: "encryptionkey@0"
I20260809 02:15:46.774500 16616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/minicluster-data/ts-0/wal/instance:
uuid: "ecef09984f904f65bdd54323d956122f"
format_stamp: "Formatted at 2026-08-09 02:15:46 on dist-test-slave-s11t"
server_key: "ec9ceb0821513b20d8bfa54cfbc4957d"
server_key_iv: "f1543de3e48b695fd5b9baa4141a71cb"
server_key_version: "encryptionkey@0"
I20260809 02:15:46.778264 16616 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 02:15:46.780575 16641 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:46.781777 16616 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 02:15:46.781930 16616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/minicluster-data/ts-0/wal
uuid: "ecef09984f904f65bdd54323d956122f"
format_stamp: "Formatted at 2026-08-09 02:15:46 on dist-test-slave-s11t"
server_key: "ec9ceb0821513b20d8bfa54cfbc4957d"
server_key_iv: "f1543de3e48b695fd5b9baa4141a71cb"
server_key_version: "encryptionkey@0"
I20260809 02:15:46.782039 16616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:15:46.802660 16616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:15:46.806152 16616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:15:46.806363 16616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:15:46.806949 16616 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:15:46.807781 16616 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:15:46.807857 16616 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:46.807924 16616 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:15:46.807973 16616 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:46.817504 16616 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.129:36261
I20260809 02:15:46.817582 16809 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.129:36261 every 8 connection(s)
I20260809 02:15:46.818408 16616 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
I20260809 02:15:46.821935 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 16616
I20260809 02:15:46.822052 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/minicluster-data/ts-0/wal/instance
I20260809 02:15:46.822369 15662 external_mini_cluster.cc:1428] Setting key c6b6c1220b7b110af2958f66d1eebf57
Aug 09 02:15:46 dist-test-slave-s11t krb5kdc[16438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241746, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.129@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.75.130@KRBTEST.COM; defaulting to no policy
I20260809 02:15:46.835064 16814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:46.820352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.129:33613 (local address 127.15.75.190:36703)
0809 02:15:46.820601 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:46.820605 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:46.821479 (+ 874us) server_negotiation.cc:407] Connection header received
0809 02:15:46.823393 (+ 1914us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:46.823397 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:46.823442 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:46.823544 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:46.824909 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:46.825856 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:46.827676 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:46.827912 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:46.828010 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:46.831112 (+ 3102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:15:46.831115 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:15:46.831117 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:15:46.831155 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:15:46.832842 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:46.833428 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:46.833432 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:46.833434 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:46.833541 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:46.833900 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:46.833903 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:46.833904 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:46.834091 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:15:46.834167 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:46.834777 (+ 610us) server_negotiation.cc:299] Negotiation successful
0809 02:15:46.834912 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":89,"threads_started":1}
I20260809 02:15:46.835894 16815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:46.820764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36703 (local address 127.15.75.129:33613)
0809 02:15:46.821295 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:46.821351 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:46.823167 (+ 1816us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:46.823718 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:46.823729 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:46.824241 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:46.824756 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:46.824781 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:46.826013 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:46.826019 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:46.827535 (+ 1516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:46.827551 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:46.828078 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:46.828085 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:46.828325 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:46.829083 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:46.829110 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:46.830920 (+ 1810us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:15:46.832964 (+ 2044us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:15:46.832971 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:15:46.832986 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:15:46.833249 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:15:46.833644 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:15:46.833646 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:15:46.833648 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:15:46.833803 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:15:46.834182 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:15:46.834187 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:15:46.834468 (+ 281us) client_negotiation.cc:769] Sending connection context
0809 02:15:46.834638 (+ 170us) client_negotiation.cc:240] Negotiation successful
0809 02:15:46.834983 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":115,"threads_started":1}
I20260809 02:15:46.837319 16811 heartbeater.cc:347] Connected to a master server at 127.15.75.190:36703
I20260809 02:15:46.837584 16811 heartbeater.cc:464] Registering TS with master...
I20260809 02:15:46.838140 16811 heartbeater.cc:511] Master 127.15.75.190:36703 requested a full tablet report, sending...
I20260809 02:15:46.840001 16505 ts_manager.cc:194] Registered new tserver with Master: ecef09984f904f65bdd54323d956122f (127.15.75.129:36261)
I20260809 02:15:46.841288 16505 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.75.129@KRBTEST.COM'} at 127.15.75.129:33613
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab.
Entry for principal kudu/127.15.75.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab.
W20260809 02:15:47.611897 15662 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.15.75.130: real 0.788s user 0.000s sys 0.004s
I20260809 02:15:47.612465 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.130:0
--local_ip_for_outbound_sockets=127.15.75.130
--webserver_interface=127.15.75.130
--webserver_port=0
--tserver_master_addrs=127.15.75.190:36703
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab
--principal=kudu/127.15.75.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:36655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:15:47.722518 16821 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:15:47.722805 16821 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:15:47.722880 16821 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:15:47.726809 16821 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:15:47.726890 16821 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:15:47.726975 16821 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.130
I20260809 02:15:47.732048 16821 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:36655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab
--principal=kudu/127.15.75.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.75.130
--webserver_port=0
--tserver_master_addrs=127.15.75.190:36703
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16821
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.130
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:15:47.733049 16821 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:15:47.734030 16821 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:15:47.743440 16835 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:15:47.743498 16821 server_base.cc:1034] running on GCE node
W20260809 02:15:47.743520 16830 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:15:47.743441 16831 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:15:47.744084 16821 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:15:47.744765 16821 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:15:47.745956 16821 hybrid_clock.cc:648] HybridClock initialized: now 1786241747745935 us; error 31 us; skew 500 ppm
Aug 09 02:15:47 dist-test-slave-s11t krb5kdc[16438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241747, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:15:47.748768 16821 init.cc:377] Logged in from keytab as kudu/127.15.75.130@KRBTEST.COM (short username kudu)
I20260809 02:15:47.749526 16821 webserver.cc:460] Webserver started at http://127.15.75.130:44343/ using document root <none> and password file <none>
I20260809 02:15:47.750017 16821 fs_manager.cc:359] Metadata directory not provided
I20260809 02:15:47.750077 16821 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:15:47.750242 16821 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:15:47.752126 16821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/minicluster-data/ts-1/data/instance:
uuid: "ced21de4a4e54e3485c8475b0de3bb0f"
format_stamp: "Formatted at 2026-08-09 02:15:47 on dist-test-slave-s11t"
server_key: "178117e8a55c08d84d2670b360fadfdd"
server_key_iv: "a066f6060d8e9f441635ed1cd4eec935"
server_key_version: "encryptionkey@0"
I20260809 02:15:47.752631 16821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/minicluster-data/ts-1/wal/instance:
uuid: "ced21de4a4e54e3485c8475b0de3bb0f"
format_stamp: "Formatted at 2026-08-09 02:15:47 on dist-test-slave-s11t"
server_key: "178117e8a55c08d84d2670b360fadfdd"
server_key_iv: "a066f6060d8e9f441635ed1cd4eec935"
server_key_version: "encryptionkey@0"
I20260809 02:15:47.756364 16821 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260809 02:15:47.758857 16844 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:47.760022 16821 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 02:15:47.760143 16821 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/minicluster-data/ts-1/wal
uuid: "ced21de4a4e54e3485c8475b0de3bb0f"
format_stamp: "Formatted at 2026-08-09 02:15:47 on dist-test-slave-s11t"
server_key: "178117e8a55c08d84d2670b360fadfdd"
server_key_iv: "a066f6060d8e9f441635ed1cd4eec935"
server_key_version: "encryptionkey@0"
I20260809 02:15:47.760248 16821 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:15:47.776482 16821 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:15:47.779627 16821 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:15:47.779841 16821 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:15:47.780414 16821 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:15:47.781241 16821 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:15:47.781289 16821 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:47.781327 16821 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:15:47.781342 16821 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:47.790366 16821 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.130:43817
I20260809 02:15:47.790396 17001 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.130:43817 every 8 connection(s)
I20260809 02:15:47.791182 16821 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
I20260809 02:15:47.799409 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 16821
I20260809 02:15:47.799503 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/minicluster-data/ts-1/wal/instance
I20260809 02:15:47.799790 15662 external_mini_cluster.cc:1428] Setting key 3dab3dc28f7622f2670c5a994ad0f5f7
Aug 09 02:15:47 dist-test-slave-s11t krb5kdc[16438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241747, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.130@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:15:47.804312 17005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:47.792946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.130:56719 (local address 127.15.75.190:36703)
0809 02:15:47.793183 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:47.793187 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:47.793879 (+ 692us) server_negotiation.cc:407] Connection header received
0809 02:15:47.795092 (+ 1213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:47.795095 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:47.795140 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:47.795222 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:47.796283 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:47.796889 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:47.797970 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:47.798190 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:47.798268 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:47.800767 (+ 2499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:15:47.800770 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:15:47.800772 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:15:47.800795 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:15:47.802266 (+ 1471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:47.802783 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:47.802788 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:47.802790 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:47.802878 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:47.803231 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:47.803234 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:47.803236 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:47.803416 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:15:47.803524 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:47.804001 (+ 477us) server_negotiation.cc:299] Negotiation successful
0809 02:15:47.804124 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":73,"threads_started":1}
I20260809 02:15:47.805217 17006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:47.793292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36703 (local address 127.15.75.130:56719)
0809 02:15:47.793711 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:47.793755 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:47.794905 (+ 1150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:47.795382 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:47.795388 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:47.795708 (+ 320us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:47.796146 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:47.796159 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:47.797001 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:47.797003 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:47.797859 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:47.797868 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:47.798315 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:47.798317 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:47.798447 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:47.799015 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:47.799031 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:47.800576 (+ 1545us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:15:47.802385 (+ 1809us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:15:47.802398 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:15:47.802412 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:15:47.802665 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:15:47.802981 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:15:47.802986 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:15:47.802988 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:15:47.803125 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:15:47.803523 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:15:47.803531 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:15:47.803768 (+ 237us) client_negotiation.cc:769] Sending connection context
0809 02:15:47.803958 (+ 190us) client_negotiation.cc:240] Negotiation successful
0809 02:15:47.804263 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":110,"threads_started":1}
I20260809 02:15:47.806345 17002 heartbeater.cc:347] Connected to a master server at 127.15.75.190:36703
I20260809 02:15:47.806602 17002 heartbeater.cc:464] Registering TS with master...
I20260809 02:15:47.807080 17002 heartbeater.cc:511] Master 127.15.75.190:36703 requested a full tablet report, sending...
I20260809 02:15:47.808130 16505 ts_manager.cc:194] Registered new tserver with Master: ced21de4a4e54e3485c8475b0de3bb0f (127.15.75.130:43817)
I20260809 02:15:47.808976 16505 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.75.130@KRBTEST.COM'} at 127.15.75.130:56719
WARNING: no policy specified for kudu/127.15.75.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab.
Entry for principal kudu/127.15.75.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab.
I20260809 02:15:47.827553 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.131:0
--local_ip_for_outbound_sockets=127.15.75.131
--webserver_interface=127.15.75.131
--webserver_port=0
--tserver_master_addrs=127.15.75.190:36703
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab
--principal=kudu/127.15.75.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:36655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 02:15:47.843914 16811 heartbeater.cc:503] Master 127.15.75.190:36703 was elected leader, sending a full tablet report...
W20260809 02:15:47.938324 17009 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:15:47.938568 17009 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:15:47.938611 17009 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:15:47.942936 17009 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:15:47.942986 17009 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:15:47.943084 17009 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.131
I20260809 02:15:47.948153 17009 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:36655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab
--principal=kudu/127.15.75.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.75.131
--webserver_port=0
--tserver_master_addrs=127.15.75.190:36703
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17009
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.131
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:15:47.949157 17009 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:15:47.950115 17009 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:15:47.959728 17024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:15:47.959741 17018 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:15:47.959748 17014 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:15:47.959811 17009 server_base.cc:1034] running on GCE node
I20260809 02:15:47.960317 17009 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:15:47.960999 17009 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:15:47.962186 17009 hybrid_clock.cc:648] HybridClock initialized: now 1786241747962174 us; error 24 us; skew 500 ppm
Aug 09 02:15:47 dist-test-slave-s11t krb5kdc[16438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241747, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:15:47.964982 17009 init.cc:377] Logged in from keytab as kudu/127.15.75.131@KRBTEST.COM (short username kudu)
I20260809 02:15:47.965811 17009 webserver.cc:460] Webserver started at http://127.15.75.131:44113/ using document root <none> and password file <none>
I20260809 02:15:47.966315 17009 fs_manager.cc:359] Metadata directory not provided
I20260809 02:15:47.966373 17009 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:15:47.966531 17009 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:15:47.968416 17009 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/minicluster-data/ts-2/data/instance:
uuid: "7af3896468a140dcb485a55d8df92f58"
format_stamp: "Formatted at 2026-08-09 02:15:47 on dist-test-slave-s11t"
server_key: "427359de93e6fb7bebaa5316d55a1de2"
server_key_iv: "73d1c51f6e1dfb26ca85025538459d01"
server_key_version: "encryptionkey@0"
I20260809 02:15:47.968981 17009 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/minicluster-data/ts-2/wal/instance:
uuid: "7af3896468a140dcb485a55d8df92f58"
format_stamp: "Formatted at 2026-08-09 02:15:47 on dist-test-slave-s11t"
server_key: "427359de93e6fb7bebaa5316d55a1de2"
server_key_iv: "73d1c51f6e1dfb26ca85025538459d01"
server_key_version: "encryptionkey@0"
I20260809 02:15:47.972435 17009 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 02:15:47.975116 17035 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:47.976336 17009 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 02:15:47.976511 17009 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/minicluster-data/ts-2/wal
uuid: "7af3896468a140dcb485a55d8df92f58"
format_stamp: "Formatted at 2026-08-09 02:15:47 on dist-test-slave-s11t"
server_key: "427359de93e6fb7bebaa5316d55a1de2"
server_key_iv: "73d1c51f6e1dfb26ca85025538459d01"
server_key_version: "encryptionkey@0"
I20260809 02:15:47.976620 17009 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:15:47.988818 17009 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:15:47.991482 17009 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:15:47.991670 17009 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:15:47.992218 17009 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:15:47.993099 17009 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:15:47.993170 17009 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:47.993229 17009 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:15:47.993276 17009 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:48.002128 17009 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.131:46707
I20260809 02:15:48.002154 17202 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.131:46707 every 8 connection(s)
I20260809 02:15:48.002972 17009 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
I20260809 02:15:48.003590 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 17009
I20260809 02:15:48.003715 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241741788589-15662-0/minicluster-data/ts-2/wal/instance
I20260809 02:15:48.004004 15662 external_mini_cluster.cc:1428] Setting key 685973f4b9ccd151c180793cff7037c8
Aug 09 02:15:48 dist-test-slave-s11t krb5kdc[16438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241747, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.131@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:15:48.016027 17005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:48.004902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.131:38803 (local address 127.15.75.190:36703)
0809 02:15:48.005022 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:48.005028 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:48.006061 (+ 1033us) server_negotiation.cc:407] Connection header received
0809 02:15:48.007434 (+ 1373us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:48.007437 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:48.007481 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:48.007547 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:48.008591 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:48.009186 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:48.010269 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:48.010493 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:48.010567 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:48.012951 (+ 2384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:15:48.012954 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:15:48.012956 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:15:48.012980 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:15:48.014282 (+ 1302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:48.014727 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:48.014730 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:48.014731 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:48.014825 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:48.015128 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:48.015130 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:48.015131 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:48.015301 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:15:48.015379 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:48.015791 (+ 412us) server_negotiation.cc:299] Negotiation successful
0809 02:15:48.015906 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 02:15:48.016885 17208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:48.005402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36703 (local address 127.15.75.131:38803)
0809 02:15:48.005922 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:48.005972 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:48.007252 (+ 1280us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:48.007673 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:48.007679 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:48.008028 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:48.008473 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:48.008486 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:48.009298 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:48.009301 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:48.010165 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:48.010175 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:48.010616 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:48.010619 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:48.010754 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:48.011272 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:48.011287 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:48.012794 (+ 1507us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:15:48.014389 (+ 1595us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:15:48.014395 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:15:48.014406 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:15:48.014640 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:15:48.014913 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:15:48.014915 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:15:48.014916 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:15:48.015047 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:15:48.015391 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:15:48.015396 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:15:48.015596 (+ 200us) client_negotiation.cc:769] Sending connection context
0809 02:15:48.015739 (+ 143us) client_negotiation.cc:240] Negotiation successful
0809 02:15:48.015976 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":148,"threads_started":1}
I20260809 02:15:48.018018 17203 heartbeater.cc:347] Connected to a master server at 127.15.75.190:36703
I20260809 02:15:48.018245 17203 heartbeater.cc:464] Registering TS with master...
I20260809 02:15:48.018666 17203 heartbeater.cc:511] Master 127.15.75.190:36703 requested a full tablet report, sending...
I20260809 02:15:48.019618 16504 ts_manager.cc:194] Registered new tserver with Master: 7af3896468a140dcb485a55d8df92f58 (127.15.75.131:46707)
I20260809 02:15:48.020186 16504 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.75.131@KRBTEST.COM'} at 127.15.75.131:38803
I20260809 02:15:48.029925 15662 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 02:15:48.040537 17005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:48.033289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44374 (local address 127.15.75.190:36703)
0809 02:15:48.033432 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:48.033437 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:48.033563 (+ 126us) server_negotiation.cc:407] Connection header received
0809 02:15:48.033730 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:48.033733 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:48.033781 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:48.033862 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:48.034554 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:48.035072 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:48.035877 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:48.036130 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:48.036239 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:48.037647 (+ 1408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:15:48.037650 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:15:48.037653 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:15:48.037678 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:15:48.038839 (+ 1161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:48.039267 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:48.039270 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:48.039272 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:48.039362 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:48.039643 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:48.039646 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:48.039648 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:48.039864 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:15:48.040005 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:48.040250 (+ 245us) server_negotiation.cc:299] Negotiation successful
0809 02:15:48.040373 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 02:15:48.045053 16504 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44374:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:15:48.047886 16504 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:15:48.062994 17224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:48.056153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.129:36261 (local address 127.0.0.1:40034)
0809 02:15:48.057242 (+ 1089us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:48.057293 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:48.057421 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:48.058117 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:48.058123 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:48.058144 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:15:48.058269 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:48.058276 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:48.059768 (+ 1492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:48.059773 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:48.061462 (+ 1689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:48.061473 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:48.062608 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:48.062615 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:48.062720 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:48.062740 (+ 20us) client_negotiation.cc:769] Sending connection context
0809 02:15:48.062831 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 02:15:48.062891 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":951,"mutex_wait_us":142,"spinlock_wait_cycles":3968,"thread_start_us":68,"threads_started":1}
I20260809 02:15:48.063330 17227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:48.056164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54014 (local address 127.15.75.131:46707)
0809 02:15:48.056552 (+ 388us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:48.056561 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:48.057666 (+ 1105us) server_negotiation.cc:407] Connection header received
0809 02:15:48.057757 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:48.057765 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:48.057975 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:48.058127 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:15:48.058642 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:48.059743 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:48.061598 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:48.062245 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:48.062378 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:48.062526 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:48.063079 (+ 553us) server_negotiation.cc:299] Negotiation successful
0809 02:15:48.063191 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":120,"threads_started":1}
I20260809 02:15:48.063687 17230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:48.057383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53460 (local address 127.15.75.130:43817)
0809 02:15:48.057743 (+ 360us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:48.057754 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:48.057772 (+ 18us) server_negotiation.cc:407] Connection header received
0809 02:15:48.057834 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:48.057838 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:48.057977 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:48.058127 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:15:48.059734 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:48.060740 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:48.061830 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:48.062598 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:48.062706 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:48.062826 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:48.063490 (+ 664us) server_negotiation.cc:299] Negotiation successful
0809 02:15:48.063584 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":97,"threads_started":1}
I20260809 02:15:48.063762 17229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:48.056858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40034 (local address 127.15.75.129:36261)
0809 02:15:48.057752 (+ 894us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:48.057758 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:48.057778 (+ 20us) server_negotiation.cc:407] Connection header received
0809 02:15:48.057838 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:48.057841 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:48.057978 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:48.058127 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:15:48.058410 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:48.059626 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:48.061621 (+ 1995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:48.062444 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:48.062555 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:48.062722 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:48.063347 (+ 625us) server_negotiation.cc:299] Negotiation successful
0809 02:15:48.063658 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":86,"threads_started":1}
I20260809 02:15:48.063390 17228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:48.056786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.130:43817 (local address 127.0.0.1:53460)
0809 02:15:48.057239 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:48.057298 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:48.057423 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:48.058290 (+ 867us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:48.058305 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:48.058316 (+ 11us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:15:48.058512 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:48.058519 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:48.060876 (+ 2357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:48.060883 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:48.061701 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:48.061711 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:48.062905 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:48.062909 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:48.063065 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:48.063091 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 02:15:48.063159 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 02:15:48.063231 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":383,"mutex_wait_us":46,"spinlock_wait_cycles":6784,"thread_start_us":88,"threads_started":1}
I20260809 02:15:48.063494 17225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:48.056028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.131:46707 (local address 127.0.0.1:54014)
0809 02:15:48.057557 (+ 1529us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:48.057568 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:48.057650 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:48.058290 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:48.058293 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:48.058311 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:15:48.058515 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:48.058519 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:48.059878 (+ 1359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:48.059881 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:48.061462 (+ 1581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:48.061473 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:48.062379 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:48.062382 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:48.062497 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:48.062533 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 02:15:48.062642 (+ 109us) client_negotiation.cc:240] Negotiation successful
0809 02:15:48.062778 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":799,"mutex_wait_us":205,"spinlock_wait_cycles":2944,"thread_start_us":118,"threads_started":1}
I20260809 02:15:48.066840 17105 tablet_service.cc:1467] Processing CreateTablet for tablet ee3053e9012f45a89799bb198400995b (DEFAULT_TABLE table=test-table [id=3b867be2c2054d43993788ebd4a83acb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:15:48.066898 16708 tablet_service.cc:1467] Processing CreateTablet for tablet ee3053e9012f45a89799bb198400995b (DEFAULT_TABLE table=test-table [id=3b867be2c2054d43993788ebd4a83acb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:15:48.066923 16908 tablet_service.cc:1467] Processing CreateTablet for tablet ee3053e9012f45a89799bb198400995b (DEFAULT_TABLE table=test-table [id=3b867be2c2054d43993788ebd4a83acb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:15:48.067909 16708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee3053e9012f45a89799bb198400995b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:15:48.067919 16908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee3053e9012f45a89799bb198400995b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:15:48.067945 17105 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee3053e9012f45a89799bb198400995b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:15:48.074522 17231 tablet_bootstrap.cc:492] T ee3053e9012f45a89799bb198400995b P 7af3896468a140dcb485a55d8df92f58: Bootstrap starting.
I20260809 02:15:48.075575 17234 tablet_bootstrap.cc:492] T ee3053e9012f45a89799bb198400995b P ced21de4a4e54e3485c8475b0de3bb0f: Bootstrap starting.
I20260809 02:15:48.075811 17233 tablet_bootstrap.cc:492] T ee3053e9012f45a89799bb198400995b P ecef09984f904f65bdd54323d956122f: Bootstrap starting.
I20260809 02:15:48.076979 17231 tablet_bootstrap.cc:654] T ee3053e9012f45a89799bb198400995b P 7af3896468a140dcb485a55d8df92f58: Neither blocks nor log segments found. Creating new log.
I20260809 02:15:48.077848 17233 tablet_bootstrap.cc:654] T ee3053e9012f45a89799bb198400995b P ecef09984f904f65bdd54323d956122f: Neither blocks nor log segments found. Creating new log.
I20260809 02:15:48.078326 17231 log.cc:824] T ee3053e9012f45a89799bb198400995b P 7af3896468a140dcb485a55d8df92f58: Log is configured to *not* fsync() on all Append() calls
I20260809 02:15:48.078367 17234 tablet_bootstrap.cc:654] T ee3053e9012f45a89799bb198400995b P ced21de4a4e54e3485c8475b0de3bb0f: Neither blocks nor log segments found. Creating new log.
I20260809 02:15:48.078929 17233 log.cc:824] T ee3053e9012f45a89799bb198400995b P ecef09984f904f65bdd54323d956122f: Log is configured to *not* fsync() on all Append() calls
I20260809 02:15:48.079192 17234 log.cc:824] T ee3053e9012f45a89799bb198400995b P ced21de4a4e54e3485c8475b0de3bb0f: Log is configured to *not* fsync() on all Append() calls
I20260809 02:15:48.080487 17231 tablet_bootstrap.cc:492] T ee3053e9012f45a89799bb198400995b P 7af3896468a140dcb485a55d8df92f58: No bootstrap required, opened a new log
I20260809 02:15:48.080717 17231 ts_tablet_manager.cc:1397] T ee3053e9012f45a89799bb198400995b P 7af3896468a140dcb485a55d8df92f58: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260809 02:15:48.080955 17233 tablet_bootstrap.cc:492] T ee3053e9012f45a89799bb198400995b P ecef09984f904f65bdd54323d956122f: No bootstrap required, opened a new log
I20260809 02:15:48.081154 17233 ts_tablet_manager.cc:1397] T ee3053e9012f45a89799bb198400995b P ecef09984f904f65bdd54323d956122f: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20260809 02:15:48.081192 17234 tablet_bootstrap.cc:492] T ee3053e9012f45a89799bb198400995b P ced21de4a4e54e3485c8475b0de3bb0f: No bootstrap required, opened a new log
I20260809 02:15:48.081359 17234 ts_tablet_manager.cc:1397] T ee3053e9012f45a89799bb198400995b P ced21de4a4e54e3485c8475b0de3bb0f: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260809 02:15:48.084546 17233 raft_consensus.cc:348] T ee3053e9012f45a89799bb198400995b P ecef09984f904f65bdd54323d956122f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7af3896468a140dcb485a55d8df92f58" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 46707 } } peers { permanent_uuid: "ecef09984f904f65bdd54323d956122f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 36261 } } peers { permanent_uuid: "ced21de4a4e54e3485c8475b0de3bb0f" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 43817 } }
I20260809 02:15:48.084597 17231 raft_consensus.cc:348] T ee3053e9012f45a89799bb198400995b P 7af3896468a140dcb485a55d8df92f58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7af3896468a140dcb485a55d8df92f58" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 46707 } } peers { permanent_uuid: "ecef09984f904f65bdd54323d956122f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 36261 } } peers { permanent_uuid: "ced21de4a4e54e3485c8475b0de3bb0f" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 43817 } }
I20260809 02:15:48.084916 17231 raft_consensus.cc:374] T ee3053e9012f45a89799bb198400995b P 7af3896468a140dcb485a55d8df92f58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:15:48.084959 17233 raft_consensus.cc:374] T ee3053e9012f45a89799bb198400995b P ecef09984f904f65bdd54323d956122f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:15:48.085018 17231 raft_consensus.cc:729] T ee3053e9012f45a89799bb198400995b P 7af3896468a140dcb485a55d8df92f58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7af3896468a140dcb485a55d8df92f58, State: Initialized, Role: FOLLOWER
I20260809 02:15:48.085088 17233 raft_consensus.cc:729] T ee3053e9012f45a89799bb198400995b P ecef09984f904f65bdd54323d956122f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecef09984f904f65bdd54323d956122f, State: Initialized, Role: FOLLOWER
I20260809 02:15:48.085095 17234 raft_consensus.cc:348] T ee3053e9012f45a89799bb198400995b P ced21de4a4e54e3485c8475b0de3bb0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7af3896468a140dcb485a55d8df92f58" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 46707 } } peers { permanent_uuid: "ecef09984f904f65bdd54323d956122f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 36261 } } peers { permanent_uuid: "ced21de4a4e54e3485c8475b0de3bb0f" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 43817 } }
I20260809 02:15:48.085377 17234 raft_consensus.cc:374] T ee3053e9012f45a89799bb198400995b P ced21de4a4e54e3485c8475b0de3bb0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:15:48.085457 17234 raft_consensus.cc:729] T ee3053e9012f45a89799bb198400995b P ced21de4a4e54e3485c8475b0de3bb0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ced21de4a4e54e3485c8475b0de3bb0f, State: Initialized, Role: FOLLOWER
I20260809 02:15:48.085577 17231 consensus_queue.cc:260] T ee3053e9012f45a89799bb198400995b P 7af3896468a140dcb485a55d8df92f58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7af3896468a140dcb485a55d8df92f58" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 46707 } } peers { permanent_uuid: "ecef09984f904f65bdd54323d956122f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 36261 } } peers { permanent_uuid: "ced21de4a4e54e3485c8475b0de3bb0f" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 43817 } }
I20260809 02:15:48.085789 17233 consensus_queue.cc:260] T ee3053e9012f45a89799bb198400995b P ecef09984f904f65bdd54323d956122f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7af3896468a140dcb485a55d8df92f58" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 46707 } } peers { permanent_uuid: "ecef09984f904f65bdd54323d956122f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 36261 } } peers { permanent_uuid: "ced21de4a4e54e3485c8475b0de3bb0f" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 43817 } }
I20260809 02:15:48.085860 17234 consensus_queue.cc:260] T ee3053e9012f45a89799bb198400995b P ced21de4a4e54e3485c8475b0de3bb0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7af3896468a140dcb485a55d8df92f58" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 46707 } } peers { permanent_uuid: "ecef09984f904f65bdd54323d956122f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 36261 } } peers { permanent_uuid: "ced21de4a4e54e3485c8475b0de3bb0f" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 43817 } }
I20260809 02:15:48.086289 17203 heartbeater.cc:503] Master 127.15.75.190:36703 was elected leader, sending a full tablet report...
I20260809 02:15:48.086813 17234 ts_tablet_manager.cc:1428] T ee3053e9012f45a89799bb198400995b P ced21de4a4e54e3485c8475b0de3bb0f: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260809 02:15:48.086839 17233 ts_tablet_manager.cc:1428] T ee3053e9012f45a89799bb198400995b P ecef09984f904f65bdd54323d956122f: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260809 02:15:48.087154 17002 heartbeater.cc:503] Master 127.15.75.190:36703 was elected leader, sending a full tablet report...
I20260809 02:15:48.087612 17231 ts_tablet_manager.cc:1428] T ee3053e9012f45a89799bb198400995b P 7af3896468a140dcb485a55d8df92f58: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260809 02:15:48.189988 17239 raft_consensus.cc:482] T ee3053e9012f45a89799bb198400995b P 7af3896468a140dcb485a55d8df92f58 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:15:48.190194 17239 raft_consensus.cc:504] T ee3053e9012f45a89799bb198400995b P 7af3896468a140dcb485a55d8df92f58 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7af3896468a140dcb485a55d8df92f58" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 46707 } } peers { permanent_uuid: "ecef09984f904f65bdd54323d956122f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 36261 } } peers { permanent_uuid: "ced21de4a4e54e3485c8475b0de3bb0f" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 43817 } }
I20260809 02:15:48.191274 17239 leader_election.cc:288] T ee3053e9012f45a89799bb198400995b P 7af3896468a140dcb485a55d8df92f58 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ecef09984f904f65bdd54323d956122f (127.15.75.129:36261), ced21de4a4e54e3485c8475b0de3bb0f (127.15.75.130:43817)
I20260809 02:15:48.195241 17208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:48.191569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.129:36261 (local address 127.15.75.131:53707)
0809 02:15:48.191693 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:48.191711 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:48.191817 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:48.192077 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:48.192080 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:48.192100 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:15:48.192247 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:48.192255 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:48.193182 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:48.193185 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:48.194078 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:48.194087 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:48.194827 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:48.194831 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:48.194940 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:48.194967 (+ 27us) client_negotiation.cc:769] Sending connection context
0809 02:15:48.195044 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 02:15:48.195112 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260809 02:15:48.195281 17229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:48.191708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.131:53707 (local address 127.15.75.129:36261)
0809 02:15:48.191828 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:48.191832 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:48.191849 (+ 17us) server_negotiation.cc:407] Connection header received
0809 02:15:48.191907 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:48.191912 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:48.191967 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:48.192050 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:15:48.192380 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:48.193072 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:48.194202 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:48.194714 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:48.194804 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:48.194844 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:48.195075 (+ 231us) server_negotiation.cc:299] Negotiation successful
0809 02:15:48.195151 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 02:15:48.195521 17247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:48.191569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.130:43817 (local address 127.15.75.131:41517)
0809 02:15:48.192100 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:48.192115 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:48.192212 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:48.192475 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:48.192478 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:48.192496 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:15:48.192647 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:48.192653 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:48.193494 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:48.193497 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:48.194358 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:48.194368 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:48.195091 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:48.195095 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:48.195191 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:48.195214 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 02:15:48.195294 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 02:15:48.195368 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"mutex_wait_us":118,"thread_start_us":168,"threads_started":1}
I20260809 02:15:48.195570 17230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:48.191795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.131:41517 (local address 127.15.75.130:43817)
0809 02:15:48.191940 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:48.191944 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:48.192210 (+ 266us) server_negotiation.cc:407] Connection header received
0809 02:15:48.192300 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:48.192306 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:48.192368 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:48.192438 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:15:48.192787 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:48.193378 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:48.194481 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:48.194958 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:48.195079 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:48.195124 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:48.195333 (+ 209us) server_negotiation.cc:299] Negotiation successful
0809 02:15:48.195418 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 02:15:48.196314 16736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee3053e9012f45a89799bb198400995b" candidate_uuid: "7af3896468a140dcb485a55d8df92f58" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ecef09984f904f65bdd54323d956122f" is_pre_election: true
I20260809 02:15:48.196660 16736 raft_consensus.cc:2454] T ee3053e9012f45a89799bb198400995b P ecef09984f904f65bdd54323d956122f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7af3896468a140dcb485a55d8df92f58 in term 0.
I20260809 02:15:48.197227 17036 leader_election.cc:302] T ee3053e9012f45a89799bb198400995b P 7af3896468a140dcb485a55d8df92f58 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7af3896468a140dcb485a55d8df92f58, ecef09984f904f65bdd54323d956122f; no voters:
I20260809 02:15:48.197520 17239 raft_consensus.cc:2781] T ee3053e9012f45a89799bb198400995b P 7af3896468a140dcb485a55d8df92f58 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:15:48.197598 17239 raft_consensus.cc:482] T ee3053e9012f45a89799bb198400995b P 7af3896468a140dcb485a55d8df92f58 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:15:48.197628 17239 raft_consensus.cc:3037] T ee3053e9012f45a89799bb198400995b P 7af3896468a140dcb485a55d8df92f58 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:15:48.198529 16935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee3053e9012f45a89799bb198400995b" candidate_uuid: "7af3896468a140dcb485a55d8df92f58" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ced21de4a4e54e3485c8475b0de3bb0f" is_pre_election: true
I20260809 02:15:48.198727 17239 raft_consensus.cc:504] T ee3053e9012f45a89799bb198400995b P 7af3896468a140dcb485a55d8df92f58 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7af3896468a140dcb485a55d8df92f58" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 46707 } } peers { permanent_uuid: "ecef09984f904f65bdd54323d956122f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 36261 } } peers { permanent_uuid: "ced21de4a4e54e3485c8475b0de3bb0f" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 43817 } }
I20260809 02:15:48.198828 16935 raft_consensus.cc:2454] T ee3053e9012f45a89799bb198400995b P ced21de4a4e54e3485c8475b0de3bb0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7af3896468a140dcb485a55d8df92f58 in term 0.
I20260809 02:15:48.199126 17239 leader_election.cc:288] T ee3053e9012f45a89799bb198400995b P 7af3896468a140dcb485a55d8df92f58 [CANDIDATE]: Term 1 election: Requested vote from peers ecef09984f904f65bdd54323d956122f (127.15.75.129:36261), ced21de4a4e54e3485c8475b0de3bb0f (127.15.75.130:43817)
I20260809 02:15:48.199505 16935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee3053e9012f45a89799bb198400995b" candidate_uuid: "7af3896468a140dcb485a55d8df92f58" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ced21de4a4e54e3485c8475b0de3bb0f"
I20260809 02:15:48.199556 16736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee3053e9012f45a89799bb198400995b" candidate_uuid: "7af3896468a140dcb485a55d8df92f58" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ecef09984f904f65bdd54323d956122f"
I20260809 02:15:48.199635 16935 raft_consensus.cc:3037] T ee3053e9012f45a89799bb198400995b P ced21de4a4e54e3485c8475b0de3bb0f [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:15:48.199694 16736 raft_consensus.cc:3037] T ee3053e9012f45a89799bb198400995b P ecef09984f904f65bdd54323d956122f [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:15:48.200662 16935 raft_consensus.cc:2454] T ee3053e9012f45a89799bb198400995b P ced21de4a4e54e3485c8475b0de3bb0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7af3896468a140dcb485a55d8df92f58 in term 1.
I20260809 02:15:48.200832 16736 raft_consensus.cc:2454] T ee3053e9012f45a89799bb198400995b P ecef09984f904f65bdd54323d956122f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7af3896468a140dcb485a55d8df92f58 in term 1.
I20260809 02:15:48.201066 17040 leader_election.cc:302] T ee3053e9012f45a89799bb198400995b P 7af3896468a140dcb485a55d8df92f58 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7af3896468a140dcb485a55d8df92f58, ced21de4a4e54e3485c8475b0de3bb0f; no voters:
I20260809 02:15:48.201236 17239 raft_consensus.cc:2781] T ee3053e9012f45a89799bb198400995b P 7af3896468a140dcb485a55d8df92f58 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:15:48.201472 17239 raft_consensus.cc:686] T ee3053e9012f45a89799bb198400995b P 7af3896468a140dcb485a55d8df92f58 [term 1 LEADER]: Becoming Leader. State: Replica: 7af3896468a140dcb485a55d8df92f58, State: Running, Role: LEADER
I20260809 02:15:48.201792 17239 consensus_queue.cc:237] T ee3053e9012f45a89799bb198400995b P 7af3896468a140dcb485a55d8df92f58 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7af3896468a140dcb485a55d8df92f58" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 46707 } } peers { permanent_uuid: "ecef09984f904f65bdd54323d956122f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 36261 } } peers { permanent_uuid: "ced21de4a4e54e3485c8475b0de3bb0f" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 43817 } }
I20260809 02:15:48.205190 16496 catalog_manager.cc:5495] T ee3053e9012f45a89799bb198400995b P 7af3896468a140dcb485a55d8df92f58 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7af3896468a140dcb485a55d8df92f58 (127.15.75.131). New cstate: current_term: 1 leader_uuid: "7af3896468a140dcb485a55d8df92f58" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7af3896468a140dcb485a55d8df92f58" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 46707 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ecef09984f904f65bdd54323d956122f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 36261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ced21de4a4e54e3485c8475b0de3bb0f" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 43817 } health_report { overall_health: UNKNOWN } } }
Aug 09 02:15:48 dist-test-slave-s11t krb5kdc[16438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241745, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.75.129@KRBTEST.COM
W20260809 02:15:48.253988 17204 tablet.cc:2369] T ee3053e9012f45a89799bb198400995b P 7af3896468a140dcb485a55d8df92f58: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:15:48.255313 17229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:48.243319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40040 (local address 127.15.75.129:36261)
0809 02:15:48.243493 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:48.243498 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:48.243633 (+ 135us) server_negotiation.cc:407] Connection header received
0809 02:15:48.243796 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:48.243799 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:48.243861 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:48.243964 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:48.244872 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:48.245816 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:48.247226 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:48.247655 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:48.247776 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:48.251198 (+ 3422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:15:48.251216 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:15:48.251237 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:15:48.251267 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:15:48.253411 (+ 2144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:48.253920 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:48.253927 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:48.253929 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:48.254021 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:48.254367 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:48.254371 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:48.254372 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:48.254638 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:15:48.254855 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:48.255020 (+ 165us) server_negotiation.cc:299] Negotiation successful
0809 02:15:48.255141 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
Aug 09 02:15:48 dist-test-slave-s11t krb5kdc[16438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241748, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 02:15:48 dist-test-slave-s11t krb5kdc[16438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241748, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.75.129@KRBTEST.COM
I20260809 02:15:48.283608 17229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:48.275449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40042 (local address 127.15.75.129:36261)
0809 02:15:48.275586 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:48.275592 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:48.275771 (+ 179us) server_negotiation.cc:407] Connection header received
0809 02:15:48.275960 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:48.275963 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:48.276005 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:48.276092 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:48.276797 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:48.277321 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:48.278166 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:48.278406 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:48.278516 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:48.280988 (+ 2472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:15:48.280991 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:15:48.280993 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:15:48.281015 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:15:48.282170 (+ 1155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:48.282597 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:48.282599 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:48.282601 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:48.282672 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:48.282918 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:48.282920 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:48.282922 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:48.283072 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:15:48.283171 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:48.283389 (+ 218us) server_negotiation.cc:299] Negotiation successful
0809 02:15:48.283490 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
W20260809 02:15:48.292223 17004 tablet.cc:2369] T ee3053e9012f45a89799bb198400995b P ced21de4a4e54e3485c8475b0de3bb0f: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 09 02:15:48 dist-test-slave-s11t krb5kdc[16438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241748, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 02:15:48 dist-test-slave-s11t krb5kdc[16438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241748, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.75.129@KRBTEST.COM
I20260809 02:15:48.311309 17229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:48.302689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40056 (local address 127.15.75.129:36261)
0809 02:15:48.302876 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:48.302880 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:48.302948 (+ 68us) server_negotiation.cc:407] Connection header received
0809 02:15:48.303137 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:48.303140 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:48.303188 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:48.303269 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:48.304018 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:48.304540 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:48.305543 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:48.305758 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:48.305933 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:48.308411 (+ 2478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:15:48.308414 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:15:48.308416 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:15:48.308439 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:15:48.309730 (+ 1291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:48.310142 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:48.310145 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:48.310146 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:48.310226 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:48.310487 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:48.310490 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:48.310491 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:48.310657 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:15:48.310759 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:48.310978 (+ 219us) server_negotiation.cc:299] Negotiation successful
0809 02:15:48.311078 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
W20260809 02:15:48.313047 16682 tablet_service.cc:2908] Rejecting scan request for tablet ee3053e9012f45a89799bb198400995b: Uninitialized: clean time has not yet been initialized
I20260809 02:15:48.315346 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 16616
I20260809 02:15:48.330293 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 16821
I20260809 02:15:48.339982 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 17009
I20260809 02:15:48.350454 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 16457
2026-08-09T02:15:48Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4064 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:15:48 dist-test-slave-s11t krb5kdc[17266](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:15:48 dist-test-slave-s11t krb5kdc[17266](info): set up 2 sockets
Aug 09 02:15:48 dist-test-slave-s11t krb5kdc[17266](info): commencing operation
krb5kdc: starting...
W20260809 02:15:49.598057 15662 mini_kdc.cc:121] Time spent starting KDC: real 1.221s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Aug 09 02:15:49 dist-test-slave-s11t krb5kdc[17266](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241749, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:15:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:15:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.75.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab.
Entry for principal kudu/127.15.75.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab.
I20260809 02:15:49.719223 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:41691
--webserver_interface=127.15.75.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab
--principal=kudu/127.15.75.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:45419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.75.190:41691
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:15:49.828204 17288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:15:49.828437 17288 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:15:49.828477 17288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:15:49.832432 17288 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:15:49.832485 17288 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 02:15:49.832501 17288 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:15:49.832520 17288 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:15:49.832536 17288 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:15:49.832597 17288 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:15:49.837693 17288 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:45419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.75.190:41691
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab
--principal=kudu/127.15.75.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:41691
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.75.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17288
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:15:49.838632 17288 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:15:49.839596 17288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:15:49.848090 17299 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:15:49.848096 17296 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:15:49.848117 17297 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:15:49.848268 17288 server_base.cc:1034] running on GCE node
I20260809 02:15:49.848929 17288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:15:49.849990 17288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:15:49.851189 17288 hybrid_clock.cc:648] HybridClock initialized: now 1786241749851178 us; error 31 us; skew 500 ppm
Aug 09 02:15:49 dist-test-slave-s11t krb5kdc[17266](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241749, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:15:49.853912 17288 init.cc:377] Logged in from keytab as kudu/127.15.75.190@KRBTEST.COM (short username kudu)
I20260809 02:15:49.854799 17288 webserver.cc:460] Webserver started at http://127.15.75.190:36301/ using document root <none> and password file <none>
I20260809 02:15:49.855342 17288 fs_manager.cc:359] Metadata directory not provided
I20260809 02:15:49.855445 17288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:15:49.855681 17288 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:15:49.857741 17288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/minicluster-data/master-0/data/instance:
uuid: "df9490defb084e519d100b3d99641ab0"
format_stamp: "Formatted at 2026-08-09 02:15:49 on dist-test-slave-s11t"
server_key: "eddfee056799b8181c5525065ae0b760"
server_key_iv: "c61d5cec3836fd92bdeca52bc0aa6732"
server_key_version: "encryptionkey@0"
I20260809 02:15:49.858312 17288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/minicluster-data/master-0/wal/instance:
uuid: "df9490defb084e519d100b3d99641ab0"
format_stamp: "Formatted at 2026-08-09 02:15:49 on dist-test-slave-s11t"
server_key: "eddfee056799b8181c5525065ae0b760"
server_key_iv: "c61d5cec3836fd92bdeca52bc0aa6732"
server_key_version: "encryptionkey@0"
I20260809 02:15:49.861922 17288 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 02:15:49.864169 17309 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:49.865345 17288 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 02:15:49.865502 17288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/minicluster-data/master-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/minicluster-data/master-0/wal
uuid: "df9490defb084e519d100b3d99641ab0"
format_stamp: "Formatted at 2026-08-09 02:15:49 on dist-test-slave-s11t"
server_key: "eddfee056799b8181c5525065ae0b760"
server_key_iv: "c61d5cec3836fd92bdeca52bc0aa6732"
server_key_version: "encryptionkey@0"
I20260809 02:15:49.865610 17288 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:15:49.879200 17288 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:15:49.881861 17288 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:15:49.882059 17288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:15:49.890149 17288 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.190:41691
I20260809 02:15:49.890163 17391 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.190:41691 every 8 connection(s)
I20260809 02:15:49.891034 17288 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
I20260809 02:15:49.893751 17392 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:15:49.895234 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 17288
I20260809 02:15:49.895330 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/minicluster-data/master-0/wal/instance
I20260809 02:15:49.895587 15662 external_mini_cluster.cc:1428] Setting key c7f5c42f4db39232367f0f2c70ca9d4a
I20260809 02:15:49.899778 17392 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df9490defb084e519d100b3d99641ab0: Bootstrap starting.
I20260809 02:15:49.902292 17392 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P df9490defb084e519d100b3d99641ab0: Neither blocks nor log segments found. Creating new log.
I20260809 02:15:49.903213 17392 log.cc:824] T 00000000000000000000000000000000 P df9490defb084e519d100b3d99641ab0: Log is configured to *not* fsync() on all Append() calls
Aug 09 02:15:49 dist-test-slave-s11t krb5kdc[17266](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241749, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:15:49.905169 17392 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df9490defb084e519d100b3d99641ab0: No bootstrap required, opened a new log
I20260809 02:15:49.907894 17392 raft_consensus.cc:348] T 00000000000000000000000000000000 P df9490defb084e519d100b3d99641ab0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df9490defb084e519d100b3d99641ab0" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 41691 } }
I20260809 02:15:49.908128 17392 raft_consensus.cc:374] T 00000000000000000000000000000000 P df9490defb084e519d100b3d99641ab0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:15:49.908221 17392 raft_consensus.cc:729] T 00000000000000000000000000000000 P df9490defb084e519d100b3d99641ab0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df9490defb084e519d100b3d99641ab0, State: Initialized, Role: FOLLOWER
I20260809 02:15:49.908631 17392 consensus_queue.cc:260] T 00000000000000000000000000000000 P df9490defb084e519d100b3d99641ab0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df9490defb084e519d100b3d99641ab0" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 41691 } }
I20260809 02:15:49.908807 17392 raft_consensus.cc:388] T 00000000000000000000000000000000 P df9490defb084e519d100b3d99641ab0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:15:49.908896 17392 raft_consensus.cc:482] T 00000000000000000000000000000000 P df9490defb084e519d100b3d99641ab0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:15:49.909001 17392 raft_consensus.cc:3037] T 00000000000000000000000000000000 P df9490defb084e519d100b3d99641ab0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:15:49.909974 17392 raft_consensus.cc:504] T 00000000000000000000000000000000 P df9490defb084e519d100b3d99641ab0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df9490defb084e519d100b3d99641ab0" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 41691 } }
I20260809 02:15:49.910300 17392 leader_election.cc:302] T 00000000000000000000000000000000 P df9490defb084e519d100b3d99641ab0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: df9490defb084e519d100b3d99641ab0; no voters:
I20260809 02:15:49.910439 17397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:49.896942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53956 (local address 127.15.75.190:41691)
0809 02:15:49.897346 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:49.897361 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:49.897395 (+ 34us) server_negotiation.cc:407] Connection header received
0809 02:15:49.898000 (+ 605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:49.898015 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:49.898331 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:49.898593 (+ 262us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:49.899506 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:49.900540 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:49.901733 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:49.902187 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:49.902321 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:49.905093 (+ 2772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:15:49.905098 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:15:49.905111 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:15:49.905136 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:15:49.907449 (+ 2313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:49.907924 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:49.907930 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:49.907934 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:49.908042 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:49.908366 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:49.908374 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:49.908375 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:49.908918 (+ 543us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:15:49.909120 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:49.909377 (+ 257us) server_negotiation.cc:299] Negotiation successful
0809 02:15:49.909659 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":253,"thread_start_us":134,"threads_started":1}
I20260809 02:15:49.910636 17392 leader_election.cc:288] T 00000000000000000000000000000000 P df9490defb084e519d100b3d99641ab0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:15:49.910751 17399 raft_consensus.cc:2781] T 00000000000000000000000000000000 P df9490defb084e519d100b3d99641ab0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:15:49.910957 17399 raft_consensus.cc:686] T 00000000000000000000000000000000 P df9490defb084e519d100b3d99641ab0 [term 1 LEADER]: Becoming Leader. State: Replica: df9490defb084e519d100b3d99641ab0, State: Running, Role: LEADER
I20260809 02:15:49.911296 17399 consensus_queue.cc:237] T 00000000000000000000000000000000 P df9490defb084e519d100b3d99641ab0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df9490defb084e519d100b3d99641ab0" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 41691 } }
I20260809 02:15:49.911880 17392 sys_catalog.cc:564] T 00000000000000000000000000000000 P df9490defb084e519d100b3d99641ab0 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:15:49.913007 17401 sys_catalog.cc:455] T 00000000000000000000000000000000 P df9490defb084e519d100b3d99641ab0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader df9490defb084e519d100b3d99641ab0. Latest consensus state: current_term: 1 leader_uuid: "df9490defb084e519d100b3d99641ab0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df9490defb084e519d100b3d99641ab0" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 41691 } } }
I20260809 02:15:49.913134 17401 sys_catalog.cc:458] T 00000000000000000000000000000000 P df9490defb084e519d100b3d99641ab0 [sys.catalog]: This master's current role is: LEADER
I20260809 02:15:49.913470 17400 sys_catalog.cc:455] T 00000000000000000000000000000000 P df9490defb084e519d100b3d99641ab0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "df9490defb084e519d100b3d99641ab0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df9490defb084e519d100b3d99641ab0" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 41691 } } }
I20260809 02:15:49.913528 17410 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:15:49.913564 17400 sys_catalog.cc:458] T 00000000000000000000000000000000 P df9490defb084e519d100b3d99641ab0 [sys.catalog]: This master's current role is: LEADER
I20260809 02:15:49.916440 17410 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:15:49.921766 17410 catalog_manager.cc:1347] Generated new cluster ID: 98af16712ea847dd8d03b6c83d70f374
I20260809 02:15:49.921849 17410 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:15:49.945968 17410 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:15:49.947007 17410 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:15:49.952903 17410 catalog_manager.cc:5868] T 00000000000000000000000000000000 P df9490defb084e519d100b3d99641ab0: Generated new TSK 0
I20260809 02:15:49.953509 17410 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.75.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab.
Entry for principal kudu/127.15.75.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab.
I20260809 02:15:49.986994 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.129:0
--local_ip_for_outbound_sockets=127.15.75.129
--webserver_interface=127.15.75.129
--webserver_port=0
--tserver_master_addrs=127.15.75.190:41691
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab
--principal=kudu/127.15.75.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:45419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:15:50.100431 17431 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:15:50.100796 17431 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:15:50.100874 17431 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:15:50.107551 17431 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:15:50.107715 17431 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:15:50.107803 17431 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 02:15:50.107870 17431 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.129
I20260809 02:15:50.114190 17431 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:45419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab
--principal=kudu/127.15.75.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.75.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.75.190:41691
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17431
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.129
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:15:50.115228 17431 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:15:50.116377 17431 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:15:50.127485 17440 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:15:50.128047 17441 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:15:50.128572 17444 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:15:50.129029 17431 server_base.cc:1034] running on GCE node
I20260809 02:15:50.129477 17431 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:15:50.130246 17431 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:15:50.131417 17431 hybrid_clock.cc:648] HybridClock initialized: now 1786241750131381 us; error 48 us; skew 500 ppm
Aug 09 02:15:50 dist-test-slave-s11t krb5kdc[17266](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241750, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:15:50.134886 17431 init.cc:377] Logged in from keytab as kudu/127.15.75.129@KRBTEST.COM (short username kudu)
I20260809 02:15:50.135754 17431 webserver.cc:460] Webserver started at http://127.15.75.129:43037/ using document root <none> and password file <none>
I20260809 02:15:50.136410 17431 fs_manager.cc:359] Metadata directory not provided
I20260809 02:15:50.136503 17431 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:15:50.136807 17431 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:15:50.138815 17431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/minicluster-data/ts-0/data/instance:
uuid: "d9c57cc0f0484098bdf002dd0e84738b"
format_stamp: "Formatted at 2026-08-09 02:15:50 on dist-test-slave-s11t"
server_key: "98f0e870cc4f6bb6930d3ae64b85fcbb"
server_key_iv: "4b05df0dd5749be5c86b64b471480818"
server_key_version: "encryptionkey@0"
I20260809 02:15:50.139387 17431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/minicluster-data/ts-0/wal/instance:
uuid: "d9c57cc0f0484098bdf002dd0e84738b"
format_stamp: "Formatted at 2026-08-09 02:15:50 on dist-test-slave-s11t"
server_key: "98f0e870cc4f6bb6930d3ae64b85fcbb"
server_key_iv: "4b05df0dd5749be5c86b64b471480818"
server_key_version: "encryptionkey@0"
I20260809 02:15:50.143311 17431 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260809 02:15:50.146087 17457 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:50.147531 17431 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 02:15:50.147689 17431 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/minicluster-data/ts-0/wal
uuid: "d9c57cc0f0484098bdf002dd0e84738b"
format_stamp: "Formatted at 2026-08-09 02:15:50 on dist-test-slave-s11t"
server_key: "98f0e870cc4f6bb6930d3ae64b85fcbb"
server_key_iv: "4b05df0dd5749be5c86b64b471480818"
server_key_version: "encryptionkey@0"
I20260809 02:15:50.147855 17431 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:15:50.167232 17431 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:15:50.170509 17431 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:15:50.170697 17431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:15:50.172070 17431 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:15:50.172116 17431 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:50.172180 17431 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:15:50.172214 17431 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 02:15:50 dist-test-slave-s11t krb5kdc[17266](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241750, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.129@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:15:50.183807 17431 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.129:34125
I20260809 02:15:50.183840 17630 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.129:34125 every 8 connection(s)
I20260809 02:15:50.184839 17431 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
I20260809 02:15:50.186431 17397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:50.172849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.129:37479 (local address 127.15.75.190:41691)
0809 02:15:50.172996 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:50.173002 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:50.173980 (+ 978us) server_negotiation.cc:407] Connection header received
0809 02:15:50.175228 (+ 1248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:50.175234 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:50.175322 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:50.175428 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:50.177094 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:50.177833 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:50.179108 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:50.179355 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:50.179473 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:50.182474 (+ 3001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:15:50.182480 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:15:50.182482 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:15:50.182514 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:15:50.184132 (+ 1618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:50.184752 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:50.184755 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:50.184757 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:50.184843 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:50.185263 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:50.185269 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:50.185271 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:50.185540 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:15:50.185644 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:50.186191 (+ 547us) server_negotiation.cc:299] Negotiation successful
0809 02:15:50.186313 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 02:15:50.187093 17468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:50.173328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:41691 (local address 127.15.75.129:37479)
0809 02:15:50.173801 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:50.173859 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:50.174990 (+ 1131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:50.175644 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:50.175655 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:50.176154 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:50.176905 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:50.176930 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:50.178010 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:50.178013 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:50.178970 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:50.178981 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:50.179526 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:50.179530 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:50.179708 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:50.180263 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:50.180312 (+ 49us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:50.182286 (+ 1974us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:15:50.184241 (+ 1955us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:15:50.184246 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:15:50.184258 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:15:50.184598 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:15:50.184959 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:15:50.184962 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:15:50.184964 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:15:50.185128 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:15:50.185699 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:15:50.185705 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:15:50.185965 (+ 260us) client_negotiation.cc:769] Sending connection context
0809 02:15:50.186107 (+ 142us) client_negotiation.cc:240] Negotiation successful
0809 02:15:50.186330 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":98,"threads_started":1}
I20260809 02:15:50.188380 17632 heartbeater.cc:347] Connected to a master server at 127.15.75.190:41691
I20260809 02:15:50.188568 17632 heartbeater.cc:464] Registering TS with master...
I20260809 02:15:50.189116 17632 heartbeater.cc:511] Master 127.15.75.190:41691 requested a full tablet report, sending...
I20260809 02:15:50.190791 17327 ts_manager.cc:194] Registered new tserver with Master: d9c57cc0f0484098bdf002dd0e84738b (127.15.75.129:34125)
I20260809 02:15:50.192209 17327 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.75.129@KRBTEST.COM'} at 127.15.75.129:37479
I20260809 02:15:50.193723 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 17431
I20260809 02:15:50.193845 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/minicluster-data/ts-0/wal/instance
I20260809 02:15:50.194136 15662 external_mini_cluster.cc:1428] Setting key b2dac25ae665419cb92710cc61afd691
I20260809 02:15:50.197996 17397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:50.190687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.129:40245 (local address 127.15.75.190:41691)
0809 02:15:50.190834 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:50.190838 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:50.191084 (+ 246us) server_negotiation.cc:407] Connection header received
0809 02:15:50.191185 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:50.191187 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:50.191225 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:50.191289 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:50.191763 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:50.192332 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:50.193340 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:50.193562 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:50.193671 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:50.194827 (+ 1156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:15:50.194833 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:15:50.194840 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:15:50.194861 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:15:50.196316 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:50.196797 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:50.196801 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:50.196804 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:50.196884 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:50.197181 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:50.197184 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:50.197186 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:50.197358 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:15:50.197468 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:50.197726 (+ 258us) server_negotiation.cc:299] Negotiation successful
0809 02:15:50.197831 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":55}
I20260809 02:15:50.198010 17642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:50.190552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:41691 (local address 127.15.75.129:40245)
0809 02:15:50.190992 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:50.191007 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:50.191106 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:50.191336 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:50.191338 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:50.191537 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:50.191651 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:50.191658 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:50.192478 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:50.192481 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:50.193215 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:50.193225 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:50.193677 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:50.193681 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:50.193809 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:50.194334 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:50.194378 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:50.194682 (+ 304us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:15:50.196436 (+ 1754us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:15:50.196441 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:15:50.196443 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:15:50.196642 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:15:50.196984 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:15:50.196987 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:15:50.196989 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:15:50.197078 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:15:50.197460 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:15:50.197463 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:15:50.197563 (+ 100us) client_negotiation.cc:769] Sending connection context
0809 02:15:50.197672 (+ 109us) client_negotiation.cc:240] Negotiation successful
0809 02:15:50.197819 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":122,"threads_started":1}
WARNING: no policy specified for kudu/127.15.75.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab.
Entry for principal kudu/127.15.75.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab.
I20260809 02:15:50.232021 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.130:0
--local_ip_for_outbound_sockets=127.15.75.130
--webserver_interface=127.15.75.130
--webserver_port=0
--tserver_master_addrs=127.15.75.190:41691
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab
--principal=kudu/127.15.75.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:45419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:15:50.347930 17645 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:15:50.348148 17645 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:15:50.348191 17645 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:15:50.352134 17645 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:15:50.352221 17645 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:15:50.352295 17645 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 02:15:50.352371 17645 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.130
I20260809 02:15:50.357651 17645 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:45419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab
--principal=kudu/127.15.75.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.75.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.75.190:41691
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17645
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.130
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:15:50.358584 17645 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:15:50.359539 17645 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:15:50.371153 17656 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:15:50.371130 17654 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:15:50.371136 17653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:15:50.371595 17645 server_base.cc:1034] running on GCE node
I20260809 02:15:50.371968 17645 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:15:50.372617 17645 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:15:50.373826 17645 hybrid_clock.cc:648] HybridClock initialized: now 1786241750373784 us; error 49 us; skew 500 ppm
Aug 09 02:15:50 dist-test-slave-s11t krb5kdc[17266](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241750, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:15:50.376730 17645 init.cc:377] Logged in from keytab as kudu/127.15.75.130@KRBTEST.COM (short username kudu)
I20260809 02:15:50.377655 17645 webserver.cc:460] Webserver started at http://127.15.75.130:34571/ using document root <none> and password file <none>
I20260809 02:15:50.378189 17645 fs_manager.cc:359] Metadata directory not provided
I20260809 02:15:50.378278 17645 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:15:50.378485 17645 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:15:50.380438 17645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/minicluster-data/ts-1/data/instance:
uuid: "c209b3ffbd01442d9a920d60e1049c35"
format_stamp: "Formatted at 2026-08-09 02:15:50 on dist-test-slave-s11t"
server_key: "e78455c81a1ef1a91a37f043a49b9156"
server_key_iv: "143e7d46af62df0a162cfa44f6164c59"
server_key_version: "encryptionkey@0"
I20260809 02:15:50.381050 17645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/minicluster-data/ts-1/wal/instance:
uuid: "c209b3ffbd01442d9a920d60e1049c35"
format_stamp: "Formatted at 2026-08-09 02:15:50 on dist-test-slave-s11t"
server_key: "e78455c81a1ef1a91a37f043a49b9156"
server_key_iv: "143e7d46af62df0a162cfa44f6164c59"
server_key_version: "encryptionkey@0"
I20260809 02:15:50.384846 17645 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260809 02:15:50.387337 17668 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:50.388785 17645 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 02:15:50.388917 17645 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/minicluster-data/ts-1/wal
uuid: "c209b3ffbd01442d9a920d60e1049c35"
format_stamp: "Formatted at 2026-08-09 02:15:50 on dist-test-slave-s11t"
server_key: "e78455c81a1ef1a91a37f043a49b9156"
server_key_iv: "143e7d46af62df0a162cfa44f6164c59"
server_key_version: "encryptionkey@0"
I20260809 02:15:50.389021 17645 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:15:50.398633 17645 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:15:50.401257 17645 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:15:50.401440 17645 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:15:50.402747 17645 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:15:50.402818 17645 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:50.402885 17645 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:15:50.402947 17645 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 02:15:50 dist-test-slave-s11t krb5kdc[17266](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241750, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.130@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:15:50.413779 17645 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.130:45793
I20260809 02:15:50.413923 17839 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.130:45793 every 8 connection(s)
I20260809 02:15:50.414784 17645 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
I20260809 02:15:50.417330 17397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:50.403586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.130:33177 (local address 127.15.75.190:41691)
0809 02:15:50.403756 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:50.403763 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:50.404511 (+ 748us) server_negotiation.cc:407] Connection header received
0809 02:15:50.405465 (+ 954us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:50.405469 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:50.405527 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:50.405614 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:50.407255 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:50.408134 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:50.410033 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:50.410417 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:50.410526 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:50.413176 (+ 2650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:15:50.413179 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:15:50.413182 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:15:50.413208 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:15:50.415245 (+ 2037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:50.415862 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:50.415865 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:50.415866 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:50.415949 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:50.416232 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:50.416235 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:50.416236 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:50.416410 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:15:50.416483 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:50.417045 (+ 562us) server_negotiation.cc:299] Negotiation successful
0809 02:15:50.417165 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 02:15:50.418006 17684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:50.403905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:41691 (local address 127.15.75.130:33177)
0809 02:15:50.404346 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:50.404395 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:50.405250 (+ 855us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:50.405936 (+ 686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:50.405945 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:50.406464 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:50.407078 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:50.407100 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:50.408307 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:50.408314 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:50.409887 (+ 1573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:50.409902 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:50.410563 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:50.410567 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:50.410759 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:50.411349 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:50.411365 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:50.413025 (+ 1660us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:15:50.415379 (+ 2354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:15:50.415385 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:15:50.415398 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:15:50.415743 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:15:50.416023 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:15:50.416025 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:15:50.416026 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:15:50.416146 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:15:50.416498 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:15:50.416503 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:15:50.416753 (+ 250us) client_negotiation.cc:769] Sending connection context
0809 02:15:50.416975 (+ 222us) client_negotiation.cc:240] Negotiation successful
0809 02:15:50.417272 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":135,"threads_started":1}
I20260809 02:15:50.418282 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 17645
I20260809 02:15:50.418403 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/minicluster-data/ts-1/wal/instance
I20260809 02:15:50.418651 15662 external_mini_cluster.cc:1428] Setting key cdae7fe23034db83301dda698eb1bb7c
I20260809 02:15:50.419683 17841 heartbeater.cc:347] Connected to a master server at 127.15.75.190:41691
I20260809 02:15:50.419835 17841 heartbeater.cc:464] Registering TS with master...
I20260809 02:15:50.420361 17841 heartbeater.cc:511] Master 127.15.75.190:41691 requested a full tablet report, sending...
I20260809 02:15:50.421412 17328 ts_manager.cc:194] Registered new tserver with Master: c209b3ffbd01442d9a920d60e1049c35 (127.15.75.130:45793)
I20260809 02:15:50.422127 17328 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.75.130@KRBTEST.COM'} at 127.15.75.130:33177
I20260809 02:15:50.428380 17397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:50.421562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.130:41675 (local address 127.15.75.190:41691)
0809 02:15:50.421653 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:50.421657 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:50.421950 (+ 293us) server_negotiation.cc:407] Connection header received
0809 02:15:50.422078 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:50.422081 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:50.422123 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:50.422198 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:50.422714 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:50.423212 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:50.424012 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:50.424219 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:50.424319 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:50.425399 (+ 1080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:15:50.425403 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:15:50.425405 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:15:50.425428 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:15:50.426782 (+ 1354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:50.427203 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:50.427208 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:50.427209 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:50.427287 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:50.427568 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:50.427573 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:50.427574 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:50.427766 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:15:50.427875 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:50.428127 (+ 252us) server_negotiation.cc:299] Negotiation successful
0809 02:15:50.428233 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 02:15:50.428401 17849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:50.421500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:41691 (local address 127.15.75.130:41675)
0809 02:15:50.421851 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:50.421868 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:50.421972 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:50.422300 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:50.422303 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:50.422498 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:50.422586 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:50.422592 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:50.423338 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:50.423341 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:50.423883 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:50.423891 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:50.424332 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:50.424336 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:50.424437 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:50.424964 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:50.424975 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:50.425265 (+ 290us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:15:50.426895 (+ 1630us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:15:50.426898 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:15:50.426903 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:15:50.427103 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:15:50.427379 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:15:50.427383 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:15:50.427385 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:15:50.427475 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:15:50.427867 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:15:50.427870 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:15:50.427962 (+ 92us) client_negotiation.cc:769] Sending connection context
0809 02:15:50.428074 (+ 112us) client_negotiation.cc:240] Negotiation successful
0809 02:15:50.428209 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":105,"threads_started":1}
WARNING: no policy specified for kudu/127.15.75.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab.
Entry for principal kudu/127.15.75.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab.
I20260809 02:15:50.447213 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.131:0
--local_ip_for_outbound_sockets=127.15.75.131
--webserver_interface=127.15.75.131
--webserver_port=0
--tserver_master_addrs=127.15.75.190:41691
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab
--principal=kudu/127.15.75.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:45419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:15:50.563905 17852 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:15:50.564141 17852 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:15:50.564222 17852 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:15:50.568212 17852 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:15:50.568264 17852 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:15:50.568333 17852 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 02:15:50.568414 17852 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.131
I20260809 02:15:50.573877 17852 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:45419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab
--principal=kudu/127.15.75.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.75.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.75.190:41691
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17852
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.131
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:15:50.574884 17852 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:15:50.575881 17852 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:15:50.585292 17858 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:15:50.585376 17862 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:15:50.585294 17860 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:15:50.585561 17852 server_base.cc:1034] running on GCE node
I20260809 02:15:50.585937 17852 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:15:50.586539 17852 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:15:50.587733 17852 hybrid_clock.cc:648] HybridClock initialized: now 1786241750587718 us; error 25 us; skew 500 ppm
Aug 09 02:15:50 dist-test-slave-s11t krb5kdc[17266](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241750, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:15:50.590548 17852 init.cc:377] Logged in from keytab as kudu/127.15.75.131@KRBTEST.COM (short username kudu)
I20260809 02:15:50.591346 17852 webserver.cc:460] Webserver started at http://127.15.75.131:42887/ using document root <none> and password file <none>
I20260809 02:15:50.591877 17852 fs_manager.cc:359] Metadata directory not provided
I20260809 02:15:50.591974 17852 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:15:50.592202 17852 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:15:50.594297 17852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/minicluster-data/ts-2/data/instance:
uuid: "e96a2b7179e24ca5a6d70d700c420da6"
format_stamp: "Formatted at 2026-08-09 02:15:50 on dist-test-slave-s11t"
server_key: "2f1d871165296ec01de142f415bb945a"
server_key_iv: "b600875235d8904bdda430d260eaeac3"
server_key_version: "encryptionkey@0"
I20260809 02:15:50.594844 17852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/minicluster-data/ts-2/wal/instance:
uuid: "e96a2b7179e24ca5a6d70d700c420da6"
format_stamp: "Formatted at 2026-08-09 02:15:50 on dist-test-slave-s11t"
server_key: "2f1d871165296ec01de142f415bb945a"
server_key_iv: "b600875235d8904bdda430d260eaeac3"
server_key_version: "encryptionkey@0"
I20260809 02:15:50.598364 17852 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260809 02:15:50.600555 17875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:50.601735 17852 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 02:15:50.601872 17852 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/minicluster-data/ts-2/wal
uuid: "e96a2b7179e24ca5a6d70d700c420da6"
format_stamp: "Formatted at 2026-08-09 02:15:50 on dist-test-slave-s11t"
server_key: "2f1d871165296ec01de142f415bb945a"
server_key_iv: "b600875235d8904bdda430d260eaeac3"
server_key_version: "encryptionkey@0"
I20260809 02:15:50.601986 17852 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:15:50.614485 17852 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:15:50.617512 17852 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:15:50.617700 17852 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:15:50.619036 17852 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:15:50.619107 17852 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:50.619169 17852 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:15:50.619220 17852 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 02:15:50 dist-test-slave-s11t krb5kdc[17266](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241750, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.131@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:15:50.630425 17852 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.131:42807
I20260809 02:15:50.630774 18042 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.131:42807 every 8 connection(s)
I20260809 02:15:50.631297 17852 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
I20260809 02:15:50.633273 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 17852
I20260809 02:15:50.633409 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/minicluster-data/ts-2/wal/instance
I20260809 02:15:50.633574 17397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:50.619885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.131:35455 (local address 127.15.75.190:41691)
0809 02:15:50.620039 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:50.620044 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:50.620940 (+ 896us) server_negotiation.cc:407] Connection header received
0809 02:15:50.621905 (+ 965us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:50.621909 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:50.621987 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:50.622050 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:50.623838 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:50.624753 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:50.626250 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:50.626490 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:50.626610 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:50.629544 (+ 2934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:15:50.629548 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:15:50.629550 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:15:50.629578 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:15:50.631283 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:50.631863 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:50.631867 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:50.631868 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:50.631972 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:50.632480 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:50.632483 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:50.632484 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:50.632656 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:15:50.632795 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:50.633279 (+ 484us) server_negotiation.cc:299] Negotiation successful
0809 02:15:50.633402 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 02:15:50.633718 15662 external_mini_cluster.cc:1428] Setting key 0537ad3b4f0344ea37cb68de3f91be70
I20260809 02:15:50.634438 17896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:50.620295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:41691 (local address 127.15.75.131:35455)
0809 02:15:50.620777 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:50.620826 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:50.621634 (+ 808us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:50.622473 (+ 839us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:50.622482 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:50.623023 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:50.623653 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:50.623674 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:50.624893 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:50.624897 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:50.626124 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:50.626134 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:50.626659 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:50.626663 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:50.626840 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:50.627367 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:50.627387 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:50.629366 (+ 1979us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:15:50.631405 (+ 2039us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:15:50.631411 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:15:50.631432 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:15:50.631749 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:15:50.632258 (+ 509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:15:50.632261 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:15:50.632263 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:15:50.632398 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:15:50.632805 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:15:50.632810 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:15:50.633023 (+ 213us) client_negotiation.cc:769] Sending connection context
0809 02:15:50.633231 (+ 208us) client_negotiation.cc:240] Negotiation successful
0809 02:15:50.633542 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":145,"threads_started":1}
I20260809 02:15:50.635787 18043 heartbeater.cc:347] Connected to a master server at 127.15.75.190:41691
I20260809 02:15:50.635957 18043 heartbeater.cc:464] Registering TS with master...
I20260809 02:15:50.636430 18043 heartbeater.cc:511] Master 127.15.75.190:41691 requested a full tablet report, sending...
I20260809 02:15:50.637506 17327 ts_manager.cc:194] Registered new tserver with Master: e96a2b7179e24ca5a6d70d700c420da6 (127.15.75.131:42807)
I20260809 02:15:50.638262 17327 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.75.131@KRBTEST.COM'} at 127.15.75.131:35455
I20260809 02:15:50.644138 18052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:50.637838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:41691 (local address 127.15.75.131:47341)
0809 02:15:50.638137 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:50.638151 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:50.638246 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:50.638497 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:50.638501 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:50.638672 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:50.638759 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:50.638765 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:50.639488 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:50.639492 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:50.640036 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:50.640043 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:50.640491 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:50.640495 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:50.640593 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:50.641026 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:50.641037 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:50.641307 (+ 270us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:15:50.642695 (+ 1388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:15:50.642700 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:15:50.642702 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:15:50.642887 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:15:50.643170 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:15:50.643174 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:15:50.643176 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:15:50.643256 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:15:50.643629 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:15:50.643633 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:15:50.643723 (+ 90us) client_negotiation.cc:769] Sending connection context
0809 02:15:50.643823 (+ 100us) client_negotiation.cc:240] Negotiation successful
0809 02:15:50.643958 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":109,"threads_started":1}
I20260809 02:15:50.644178 17397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:50.637944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.131:47341 (local address 127.15.75.190:41691)
0809 02:15:50.638058 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:50.638064 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:50.638223 (+ 159us) server_negotiation.cc:407] Connection header received
0809 02:15:50.638326 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:50.638328 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:50.638395 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:50.638489 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:50.638875 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:50.639363 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:50.640163 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:50.640379 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:50.640477 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:50.641459 (+ 982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:15:50.641462 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:15:50.641465 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:15:50.641488 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:15:50.642573 (+ 1085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:50.642989 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:50.642994 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:50.642995 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:50.643078 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:50.643350 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:50.643355 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:50.643357 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:50.643525 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:15:50.643628 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:50.643901 (+ 273us) server_negotiation.cc:299] Negotiation successful
0809 02:15:50.644003 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 02:15:50.647467 15662 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 02:15:50 dist-test-slave-s11t krb5kdc[17266](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241750, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 02:15:50 dist-test-slave-s11t krb5kdc[17266](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241750, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:15:50.674798 17397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:50.666229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53964 (local address 127.15.75.190:41691)
0809 02:15:50.666358 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:50.666362 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:50.666373 (+ 11us) server_negotiation.cc:407] Connection header received
0809 02:15:50.666411 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:50.666413 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:50.666451 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:50.666518 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:50.667213 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:50.667728 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:50.668579 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:50.668848 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:50.668928 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:50.671591 (+ 2663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:15:50.671595 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:15:50.671598 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:15:50.671625 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:15:50.673138 (+ 1513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:50.673558 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:50.673563 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:50.673565 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:50.673648 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:50.673944 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:50.673948 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:50.673949 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:50.674124 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:15:50.674265 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:50.674507 (+ 242us) server_negotiation.cc:299] Negotiation successful
0809 02:15:50.674624 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 02:15:50.677573 17327 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53964:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:15:50.680550 17327 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:15:50.695763 18079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:50.689630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.130:45793 (local address 127.0.0.1:49748)
0809 02:15:50.690786 (+ 1156us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:50.690799 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:50.690880 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:50.691298 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:50.691302 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:50.691320 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:15:50.691425 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:50.691429 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:50.692731 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:50.692737 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:50.694118 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:50.694127 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:50.695242 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:50.695246 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:50.695348 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:50.695392 (+ 44us) client_negotiation.cc:769] Sending connection context
0809 02:15:50.695501 (+ 109us) client_negotiation.cc:240] Negotiation successful
0809 02:15:50.695615 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1087,"thread_start_us":98,"threads_started":1}
I20260809 02:15:50.695775 18081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:50.690320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49748 (local address 127.15.75.130:45793)
0809 02:15:50.690893 (+ 573us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:50.690902 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:50.690941 (+ 39us) server_negotiation.cc:407] Connection header received
0809 02:15:50.691005 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:50.691010 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:50.691167 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:50.691283 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:15:50.691546 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:50.692601 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:50.694264 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:50.695099 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:50.695211 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:50.695400 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:50.695529 (+ 129us) server_negotiation.cc:299] Negotiation successful
0809 02:15:50.695615 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":477,"thread_start_us":136,"threads_started":1}
I20260809 02:15:50.696153 18073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:50.689454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.131:42807 (local address 127.0.0.1:48552)
0809 02:15:50.690374 (+ 920us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:50.690429 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:50.690561 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:50.692364 (+ 1803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:50.692366 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:50.692379 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:15:50.692451 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:50.692455 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:50.693867 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:50.693870 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:50.694803 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:50.694812 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:50.695745 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:50.695749 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:50.695834 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:50.695858 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 02:15:50.695921 (+ 63us) client_negotiation.cc:240] Negotiation successful
0809 02:15:50.695991 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":736,"mutex_wait_us":26,"thread_start_us":99,"threads_started":1}
I20260809 02:15:50.696724 18074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:50.689587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48552 (local address 127.15.75.131:42807)
0809 02:15:50.691968 (+ 2381us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:50.691979 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:50.692000 (+ 21us) server_negotiation.cc:407] Connection header received
0809 02:15:50.692077 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:50.692081 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:50.692246 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:50.692385 (+ 139us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:15:50.692519 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:50.693744 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:50.694933 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:50.695606 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:50.696254 (+ 648us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:50.696411 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:50.696461 (+ 50us) server_negotiation.cc:299] Negotiation successful
0809 02:15:50.696545 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2280,"thread_start_us":96,"threads_started":1}
I20260809 02:15:50.696810 18077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:50.689813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.129:34125 (local address 127.0.0.1:46764)
0809 02:15:50.691527 (+ 1714us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:50.691538 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:50.691647 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:50.692112 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:50.692115 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:50.692135 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:15:50.692221 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:50.692227 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:50.693907 (+ 1680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:50.693910 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:50.695076 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:50.695088 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:50.696371 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:50.696376 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:50.696467 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:50.696502 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 02:15:50.696573 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 02:15:50.696649 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1660,"mutex_wait_us":53,"thread_start_us":84,"threads_started":1}
I20260809 02:15:50.697006 18080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:50.690296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46764 (local address 127.15.75.129:34125)
0809 02:15:50.690684 (+ 388us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:50.690693 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:50.691631 (+ 938us) server_negotiation.cc:407] Connection header received
0809 02:15:50.691761 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:50.691766 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:50.691962 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:50.692114 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:15:50.692342 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:50.693806 (+ 1464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:50.695213 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:50.696014 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:50.696115 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:50.696258 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:50.696733 (+ 475us) server_negotiation.cc:299] Negotiation successful
0809 02:15:50.696835 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":88,"threads_started":1}
I20260809 02:15:50.699415 17936 tablet_service.cc:1467] Processing CreateTablet for tablet 32a3094951154cd28eab69b16fbf002e (DEFAULT_TABLE table=test-table [id=744891b29e0e444c9b2e7971359ead7c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:15:50.699470 17740 tablet_service.cc:1467] Processing CreateTablet for tablet 32a3094951154cd28eab69b16fbf002e (DEFAULT_TABLE table=test-table [id=744891b29e0e444c9b2e7971359ead7c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:15:50.699452 17535 tablet_service.cc:1467] Processing CreateTablet for tablet 32a3094951154cd28eab69b16fbf002e (DEFAULT_TABLE table=test-table [id=744891b29e0e444c9b2e7971359ead7c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:15:50.700402 17936 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32a3094951154cd28eab69b16fbf002e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:15:50.700436 17535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32a3094951154cd28eab69b16fbf002e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:15:50.700441 17740 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32a3094951154cd28eab69b16fbf002e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:15:50.707219 18082 tablet_bootstrap.cc:492] T 32a3094951154cd28eab69b16fbf002e P e96a2b7179e24ca5a6d70d700c420da6: Bootstrap starting.
I20260809 02:15:50.708360 18083 tablet_bootstrap.cc:492] T 32a3094951154cd28eab69b16fbf002e P d9c57cc0f0484098bdf002dd0e84738b: Bootstrap starting.
I20260809 02:15:50.708583 18084 tablet_bootstrap.cc:492] T 32a3094951154cd28eab69b16fbf002e P c209b3ffbd01442d9a920d60e1049c35: Bootstrap starting.
I20260809 02:15:50.710386 18083 tablet_bootstrap.cc:654] T 32a3094951154cd28eab69b16fbf002e P d9c57cc0f0484098bdf002dd0e84738b: Neither blocks nor log segments found. Creating new log.
I20260809 02:15:50.710386 18082 tablet_bootstrap.cc:654] T 32a3094951154cd28eab69b16fbf002e P e96a2b7179e24ca5a6d70d700c420da6: Neither blocks nor log segments found. Creating new log.
I20260809 02:15:50.710700 18084 tablet_bootstrap.cc:654] T 32a3094951154cd28eab69b16fbf002e P c209b3ffbd01442d9a920d60e1049c35: Neither blocks nor log segments found. Creating new log.
I20260809 02:15:50.711383 18082 log.cc:824] T 32a3094951154cd28eab69b16fbf002e P e96a2b7179e24ca5a6d70d700c420da6: Log is configured to *not* fsync() on all Append() calls
I20260809 02:15:50.711388 18084 log.cc:824] T 32a3094951154cd28eab69b16fbf002e P c209b3ffbd01442d9a920d60e1049c35: Log is configured to *not* fsync() on all Append() calls
I20260809 02:15:50.711683 18083 log.cc:824] T 32a3094951154cd28eab69b16fbf002e P d9c57cc0f0484098bdf002dd0e84738b: Log is configured to *not* fsync() on all Append() calls
I20260809 02:15:50.713410 18082 tablet_bootstrap.cc:492] T 32a3094951154cd28eab69b16fbf002e P e96a2b7179e24ca5a6d70d700c420da6: No bootstrap required, opened a new log
I20260809 02:15:50.713414 18084 tablet_bootstrap.cc:492] T 32a3094951154cd28eab69b16fbf002e P c209b3ffbd01442d9a920d60e1049c35: No bootstrap required, opened a new log
I20260809 02:15:50.713601 18084 ts_tablet_manager.cc:1397] T 32a3094951154cd28eab69b16fbf002e P c209b3ffbd01442d9a920d60e1049c35: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20260809 02:15:50.713601 18082 ts_tablet_manager.cc:1397] T 32a3094951154cd28eab69b16fbf002e P e96a2b7179e24ca5a6d70d700c420da6: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.004s
I20260809 02:15:50.713621 18083 tablet_bootstrap.cc:492] T 32a3094951154cd28eab69b16fbf002e P d9c57cc0f0484098bdf002dd0e84738b: No bootstrap required, opened a new log
I20260809 02:15:50.713819 18083 ts_tablet_manager.cc:1397] T 32a3094951154cd28eab69b16fbf002e P d9c57cc0f0484098bdf002dd0e84738b: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 02:15:50.717013 18083 raft_consensus.cc:348] T 32a3094951154cd28eab69b16fbf002e P d9c57cc0f0484098bdf002dd0e84738b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e96a2b7179e24ca5a6d70d700c420da6" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 42807 } } peers { permanent_uuid: "c209b3ffbd01442d9a920d60e1049c35" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45793 } } peers { permanent_uuid: "d9c57cc0f0484098bdf002dd0e84738b" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 34125 } }
I20260809 02:15:50.717271 18083 raft_consensus.cc:374] T 32a3094951154cd28eab69b16fbf002e P d9c57cc0f0484098bdf002dd0e84738b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:15:50.717347 18083 raft_consensus.cc:729] T 32a3094951154cd28eab69b16fbf002e P d9c57cc0f0484098bdf002dd0e84738b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9c57cc0f0484098bdf002dd0e84738b, State: Initialized, Role: FOLLOWER
I20260809 02:15:50.717522 18082 raft_consensus.cc:348] T 32a3094951154cd28eab69b16fbf002e P e96a2b7179e24ca5a6d70d700c420da6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e96a2b7179e24ca5a6d70d700c420da6" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 42807 } } peers { permanent_uuid: "c209b3ffbd01442d9a920d60e1049c35" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45793 } } peers { permanent_uuid: "d9c57cc0f0484098bdf002dd0e84738b" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 34125 } }
I20260809 02:15:50.717770 18082 raft_consensus.cc:374] T 32a3094951154cd28eab69b16fbf002e P e96a2b7179e24ca5a6d70d700c420da6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:15:50.717805 18083 consensus_queue.cc:260] T 32a3094951154cd28eab69b16fbf002e P d9c57cc0f0484098bdf002dd0e84738b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e96a2b7179e24ca5a6d70d700c420da6" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 42807 } } peers { permanent_uuid: "c209b3ffbd01442d9a920d60e1049c35" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45793 } } peers { permanent_uuid: "d9c57cc0f0484098bdf002dd0e84738b" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 34125 } }
I20260809 02:15:50.717877 18082 raft_consensus.cc:729] T 32a3094951154cd28eab69b16fbf002e P e96a2b7179e24ca5a6d70d700c420da6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e96a2b7179e24ca5a6d70d700c420da6, State: Initialized, Role: FOLLOWER
I20260809 02:15:50.717938 18084 raft_consensus.cc:348] T 32a3094951154cd28eab69b16fbf002e P c209b3ffbd01442d9a920d60e1049c35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e96a2b7179e24ca5a6d70d700c420da6" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 42807 } } peers { permanent_uuid: "c209b3ffbd01442d9a920d60e1049c35" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45793 } } peers { permanent_uuid: "d9c57cc0f0484098bdf002dd0e84738b" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 34125 } }
I20260809 02:15:50.718230 18084 raft_consensus.cc:374] T 32a3094951154cd28eab69b16fbf002e P c209b3ffbd01442d9a920d60e1049c35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:15:50.718322 18084 raft_consensus.cc:729] T 32a3094951154cd28eab69b16fbf002e P c209b3ffbd01442d9a920d60e1049c35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c209b3ffbd01442d9a920d60e1049c35, State: Initialized, Role: FOLLOWER
I20260809 02:15:50.718317 18082 consensus_queue.cc:260] T 32a3094951154cd28eab69b16fbf002e P e96a2b7179e24ca5a6d70d700c420da6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e96a2b7179e24ca5a6d70d700c420da6" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 42807 } } peers { permanent_uuid: "c209b3ffbd01442d9a920d60e1049c35" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45793 } } peers { permanent_uuid: "d9c57cc0f0484098bdf002dd0e84738b" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 34125 } }
I20260809 02:15:50.718485 17632 heartbeater.cc:503] Master 127.15.75.190:41691 was elected leader, sending a full tablet report...
I20260809 02:15:50.718876 18083 ts_tablet_manager.cc:1428] T 32a3094951154cd28eab69b16fbf002e P d9c57cc0f0484098bdf002dd0e84738b: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260809 02:15:50.718808 18084 consensus_queue.cc:260] T 32a3094951154cd28eab69b16fbf002e P c209b3ffbd01442d9a920d60e1049c35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e96a2b7179e24ca5a6d70d700c420da6" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 42807 } } peers { permanent_uuid: "c209b3ffbd01442d9a920d60e1049c35" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45793 } } peers { permanent_uuid: "d9c57cc0f0484098bdf002dd0e84738b" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 34125 } }
I20260809 02:15:50.719396 18082 ts_tablet_manager.cc:1428] T 32a3094951154cd28eab69b16fbf002e P e96a2b7179e24ca5a6d70d700c420da6: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 02:15:50.719686 18084 ts_tablet_manager.cc:1428] T 32a3094951154cd28eab69b16fbf002e P c209b3ffbd01442d9a920d60e1049c35: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 02:15:50.719756 18043 heartbeater.cc:503] Master 127.15.75.190:41691 was elected leader, sending a full tablet report...
I20260809 02:15:50.720009 17841 heartbeater.cc:503] Master 127.15.75.190:41691 was elected leader, sending a full tablet report...
I20260809 02:15:50.832286 18094 raft_consensus.cc:482] T 32a3094951154cd28eab69b16fbf002e P c209b3ffbd01442d9a920d60e1049c35 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:15:50.832510 18094 raft_consensus.cc:504] T 32a3094951154cd28eab69b16fbf002e P c209b3ffbd01442d9a920d60e1049c35 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e96a2b7179e24ca5a6d70d700c420da6" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 42807 } } peers { permanent_uuid: "c209b3ffbd01442d9a920d60e1049c35" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45793 } } peers { permanent_uuid: "d9c57cc0f0484098bdf002dd0e84738b" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 34125 } }
I20260809 02:15:50.833654 18094 leader_election.cc:288] T 32a3094951154cd28eab69b16fbf002e P c209b3ffbd01442d9a920d60e1049c35 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e96a2b7179e24ca5a6d70d700c420da6 (127.15.75.131:42807), d9c57cc0f0484098bdf002dd0e84738b (127.15.75.129:34125)
I20260809 02:15:50.838032 18080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:50.834039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.130:38283 (local address 127.15.75.129:34125)
0809 02:15:50.834195 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:50.834201 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:50.834598 (+ 397us) server_negotiation.cc:407] Connection header received
0809 02:15:50.834695 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:50.834697 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:50.834747 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:50.834838 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:15:50.835135 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:50.835683 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:50.836886 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:50.837488 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:50.837581 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:50.837622 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:50.837844 (+ 222us) server_negotiation.cc:299] Negotiation successful
0809 02:15:50.837901 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 02:15:50.838079 18095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:50.833932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.129:34125 (local address 127.15.75.130:38283)
0809 02:15:50.834495 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:50.834510 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:50.834609 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:50.834872 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:50.834875 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:50.834892 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:15:50.835021 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:50.835027 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:50.835808 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:50.835810 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:50.836762 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:50.836772 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:50.837602 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:50.837606 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:50.837716 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:50.837742 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 02:15:50.837821 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 02:15:50.837888 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"mutex_wait_us":168,"thread_start_us":145,"threads_started":1}
I20260809 02:15:50.838510 17684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:50.833932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.131:42807 (local address 127.15.75.130:54967)
0809 02:15:50.834093 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:50.834111 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:50.834223 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:50.834605 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:50.834609 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:50.834632 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:15:50.834762 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:50.834772 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:50.835620 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:50.835623 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:50.837262 (+ 1639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:50.837272 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:50.838046 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:50.838053 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:50.838181 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:50.838215 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 02:15:50.838303 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 02:15:50.838383 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260809 02:15:50.838531 18074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:50.834183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.130:54967 (local address 127.15.75.131:42807)
0809 02:15:50.834332 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:50.834337 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:50.834354 (+ 17us) server_negotiation.cc:407] Connection header received
0809 02:15:50.834412 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:50.834415 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:50.834467 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:50.834561 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:15:50.834888 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:50.835507 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:50.837384 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:50.837923 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:50.838031 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:50.838080 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:50.838331 (+ 251us) server_negotiation.cc:299] Negotiation successful
0809 02:15:50.838405 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 02:15:50.838822 17565 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32a3094951154cd28eab69b16fbf002e" candidate_uuid: "c209b3ffbd01442d9a920d60e1049c35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9c57cc0f0484098bdf002dd0e84738b" is_pre_election: true
I20260809 02:15:50.839116 17971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32a3094951154cd28eab69b16fbf002e" candidate_uuid: "c209b3ffbd01442d9a920d60e1049c35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e96a2b7179e24ca5a6d70d700c420da6" is_pre_election: true
I20260809 02:15:50.839152 17565 raft_consensus.cc:2454] T 32a3094951154cd28eab69b16fbf002e P d9c57cc0f0484098bdf002dd0e84738b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c209b3ffbd01442d9a920d60e1049c35 in term 0.
I20260809 02:15:50.839383 17971 raft_consensus.cc:2454] T 32a3094951154cd28eab69b16fbf002e P e96a2b7179e24ca5a6d70d700c420da6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c209b3ffbd01442d9a920d60e1049c35 in term 0.
I20260809 02:15:50.839712 17670 leader_election.cc:302] T 32a3094951154cd28eab69b16fbf002e P c209b3ffbd01442d9a920d60e1049c35 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c209b3ffbd01442d9a920d60e1049c35, d9c57cc0f0484098bdf002dd0e84738b; no voters:
I20260809 02:15:50.840026 18094 raft_consensus.cc:2781] T 32a3094951154cd28eab69b16fbf002e P c209b3ffbd01442d9a920d60e1049c35 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:15:50.840102 18094 raft_consensus.cc:482] T 32a3094951154cd28eab69b16fbf002e P c209b3ffbd01442d9a920d60e1049c35 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:15:50.840132 18094 raft_consensus.cc:3037] T 32a3094951154cd28eab69b16fbf002e P c209b3ffbd01442d9a920d60e1049c35 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:15:50.841451 18094 raft_consensus.cc:504] T 32a3094951154cd28eab69b16fbf002e P c209b3ffbd01442d9a920d60e1049c35 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e96a2b7179e24ca5a6d70d700c420da6" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 42807 } } peers { permanent_uuid: "c209b3ffbd01442d9a920d60e1049c35" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45793 } } peers { permanent_uuid: "d9c57cc0f0484098bdf002dd0e84738b" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 34125 } }
I20260809 02:15:50.841832 18094 leader_election.cc:288] T 32a3094951154cd28eab69b16fbf002e P c209b3ffbd01442d9a920d60e1049c35 [CANDIDATE]: Term 1 election: Requested vote from peers e96a2b7179e24ca5a6d70d700c420da6 (127.15.75.131:42807), d9c57cc0f0484098bdf002dd0e84738b (127.15.75.129:34125)
I20260809 02:15:50.842132 17971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32a3094951154cd28eab69b16fbf002e" candidate_uuid: "c209b3ffbd01442d9a920d60e1049c35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e96a2b7179e24ca5a6d70d700c420da6"
I20260809 02:15:50.842231 17565 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32a3094951154cd28eab69b16fbf002e" candidate_uuid: "c209b3ffbd01442d9a920d60e1049c35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9c57cc0f0484098bdf002dd0e84738b"
I20260809 02:15:50.842244 17971 raft_consensus.cc:3037] T 32a3094951154cd28eab69b16fbf002e P e96a2b7179e24ca5a6d70d700c420da6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:15:50.842379 17565 raft_consensus.cc:3037] T 32a3094951154cd28eab69b16fbf002e P d9c57cc0f0484098bdf002dd0e84738b [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:15:50.843461 17565 raft_consensus.cc:2454] T 32a3094951154cd28eab69b16fbf002e P d9c57cc0f0484098bdf002dd0e84738b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c209b3ffbd01442d9a920d60e1049c35 in term 1.
I20260809 02:15:50.843750 17971 raft_consensus.cc:2454] T 32a3094951154cd28eab69b16fbf002e P e96a2b7179e24ca5a6d70d700c420da6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c209b3ffbd01442d9a920d60e1049c35 in term 1.
I20260809 02:15:50.843775 17670 leader_election.cc:302] T 32a3094951154cd28eab69b16fbf002e P c209b3ffbd01442d9a920d60e1049c35 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c209b3ffbd01442d9a920d60e1049c35, d9c57cc0f0484098bdf002dd0e84738b; no voters:
I20260809 02:15:50.843945 18094 raft_consensus.cc:2781] T 32a3094951154cd28eab69b16fbf002e P c209b3ffbd01442d9a920d60e1049c35 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:15:50.844164 18094 raft_consensus.cc:686] T 32a3094951154cd28eab69b16fbf002e P c209b3ffbd01442d9a920d60e1049c35 [term 1 LEADER]: Becoming Leader. State: Replica: c209b3ffbd01442d9a920d60e1049c35, State: Running, Role: LEADER
I20260809 02:15:50.844789 18094 consensus_queue.cc:237] T 32a3094951154cd28eab69b16fbf002e P c209b3ffbd01442d9a920d60e1049c35 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e96a2b7179e24ca5a6d70d700c420da6" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 42807 } } peers { permanent_uuid: "c209b3ffbd01442d9a920d60e1049c35" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45793 } } peers { permanent_uuid: "d9c57cc0f0484098bdf002dd0e84738b" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 34125 } }
I20260809 02:15:50.848112 17328 catalog_manager.cc:5495] T 32a3094951154cd28eab69b16fbf002e P c209b3ffbd01442d9a920d60e1049c35 reported cstate change: term changed from 0 to 1, leader changed from <none> to c209b3ffbd01442d9a920d60e1049c35 (127.15.75.130). New cstate: current_term: 1 leader_uuid: "c209b3ffbd01442d9a920d60e1049c35" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e96a2b7179e24ca5a6d70d700c420da6" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 42807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c209b3ffbd01442d9a920d60e1049c35" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45793 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9c57cc0f0484098bdf002dd0e84738b" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 34125 } health_report { overall_health: UNKNOWN } } }
W20260809 02:15:50.882337 18044 tablet.cc:2369] T 32a3094951154cd28eab69b16fbf002e P e96a2b7179e24ca5a6d70d700c420da6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:15:50.892359 18081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:50.888420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49762 (local address 127.15.75.130:45793)
0809 02:15:50.888558 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:50.888568 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:50.888582 (+ 14us) server_negotiation.cc:407] Connection header received
0809 02:15:50.888814 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:50.888816 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:50.888879 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:50.888976 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:15:50.889209 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:50.889856 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:50.890722 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:50.891080 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:50.891183 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:50.891451 (+ 268us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:15:50.891980 (+ 529us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:15:50.892095 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:50.892169 (+ 74us) server_negotiation.cc:299] Negotiation successful
0809 02:15:50.892228 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 02:15:50.898901 17565 raft_consensus.cc:1261] T 32a3094951154cd28eab69b16fbf002e P d9c57cc0f0484098bdf002dd0e84738b [term 1 FOLLOWER]: Refusing update from remote peer c209b3ffbd01442d9a920d60e1049c35: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:15:50.899605 18099 consensus_queue.cc:1035] T 32a3094951154cd28eab69b16fbf002e P c209b3ffbd01442d9a920d60e1049c35 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9c57cc0f0484098bdf002dd0e84738b" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 34125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:15:50.901298 17971 raft_consensus.cc:1261] T 32a3094951154cd28eab69b16fbf002e P e96a2b7179e24ca5a6d70d700c420da6 [term 1 FOLLOWER]: Refusing update from remote peer c209b3ffbd01442d9a920d60e1049c35: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:15:50.901876 18094 consensus_queue.cc:1035] T 32a3094951154cd28eab69b16fbf002e P c209b3ffbd01442d9a920d60e1049c35 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e96a2b7179e24ca5a6d70d700c420da6" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 42807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:15:50.907778 18104 mvcc.cc:205] Tried to move back new op lower bound from 7316446211673366528 to 7316446211463278592. Current Snapshot: MvccSnapshot[applied={T|T < 7316446211673366528}]
I20260809 02:15:50.908385 18105 mvcc.cc:205] Tried to move back new op lower bound from 7316446211673366528 to 7316446211463278592. Current Snapshot: MvccSnapshot[applied={T|T < 7316446211673366528}]
I20260809 02:15:50.911196 18107 mvcc.cc:205] Tried to move back new op lower bound from 7316446211673366528 to 7316446211463278592. Current Snapshot: MvccSnapshot[applied={T|T < 7316446211673366528}]
I20260809 02:15:50.921103 17326 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53964:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:15:50.921357 17326 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53964:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:15:50.924018 17326 catalog_manager.cc:5782] T 00000000000000000000000000000000 P df9490defb084e519d100b3d99641ab0: Sending DeleteTablet for 3 replicas of tablet 32a3094951154cd28eab69b16fbf002e
I20260809 02:15:50.924827 17740 tablet_service.cc:1514] Processing DeleteTablet for tablet 32a3094951154cd28eab69b16fbf002e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:15:50 UTC) from {username='kudu', principal='kudu/127.15.75.190@KRBTEST.COM'} at 127.0.0.1:49748
I20260809 02:15:50.925002 17535 tablet_service.cc:1514] Processing DeleteTablet for tablet 32a3094951154cd28eab69b16fbf002e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:15:50 UTC) from {username='kudu', principal='kudu/127.15.75.190@KRBTEST.COM'} at 127.0.0.1:46764
I20260809 02:15:50.924999 17936 tablet_service.cc:1514] Processing DeleteTablet for tablet 32a3094951154cd28eab69b16fbf002e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:15:50 UTC) from {username='kudu', principal='kudu/127.15.75.190@KRBTEST.COM'} at 127.0.0.1:48552
I20260809 02:15:50.925479 18120 tablet_replica.cc:330] T 32a3094951154cd28eab69b16fbf002e P c209b3ffbd01442d9a920d60e1049c35: stopping tablet replica
I20260809 02:15:50.925486 18121 tablet_replica.cc:330] T 32a3094951154cd28eab69b16fbf002e P d9c57cc0f0484098bdf002dd0e84738b: stopping tablet replica
I20260809 02:15:50.925810 18120 raft_consensus.cc:2229] T 32a3094951154cd28eab69b16fbf002e P c209b3ffbd01442d9a920d60e1049c35 [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:15:50.925810 18121 raft_consensus.cc:2229] T 32a3094951154cd28eab69b16fbf002e P d9c57cc0f0484098bdf002dd0e84738b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:15:50.926105 18121 raft_consensus.cc:2258] T 32a3094951154cd28eab69b16fbf002e P d9c57cc0f0484098bdf002dd0e84738b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:15:50.926234 18120 raft_consensus.cc:2258] T 32a3094951154cd28eab69b16fbf002e P c209b3ffbd01442d9a920d60e1049c35 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:15:50.926430 17326 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53964:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:15:50.926843 17326 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:15:50.927667 18122 tablet_replica.cc:330] T 32a3094951154cd28eab69b16fbf002e P e96a2b7179e24ca5a6d70d700c420da6: stopping tablet replica
I20260809 02:15:50.927835 18120 ts_tablet_manager.cc:1905] T 32a3094951154cd28eab69b16fbf002e P c209b3ffbd01442d9a920d60e1049c35: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:15:50.928030 18122 raft_consensus.cc:2229] T 32a3094951154cd28eab69b16fbf002e P e96a2b7179e24ca5a6d70d700c420da6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:15:50.928385 18122 raft_consensus.cc:2258] T 32a3094951154cd28eab69b16fbf002e P e96a2b7179e24ca5a6d70d700c420da6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:15:50.930051 18121 ts_tablet_manager.cc:1905] T 32a3094951154cd28eab69b16fbf002e P d9c57cc0f0484098bdf002dd0e84738b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:15:50.931474 18120 ts_tablet_manager.cc:1918] T 32a3094951154cd28eab69b16fbf002e P c209b3ffbd01442d9a920d60e1049c35: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:15:50.931627 18120 log.cc:1196] T 32a3094951154cd28eab69b16fbf002e P c209b3ffbd01442d9a920d60e1049c35: Deleting WAL directory at /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/minicluster-data/ts-1/wal/wals/32a3094951154cd28eab69b16fbf002e
I20260809 02:15:50.932013 18120 ts_tablet_manager.cc:1939] T 32a3094951154cd28eab69b16fbf002e P c209b3ffbd01442d9a920d60e1049c35: Deleting consensus metadata
I20260809 02:15:50.933197 17312 catalog_manager.cc:4841] TS c209b3ffbd01442d9a920d60e1049c35 (127.15.75.130:45793): tablet 32a3094951154cd28eab69b16fbf002e (table test-table [id=744891b29e0e444c9b2e7971359ead7c]) successfully deleted
I20260809 02:15:50.933302 17535 tablet_service.cc:1467] Processing CreateTablet for tablet d97e0d08daf642b3b123d9293bffba89 (DEFAULT_TABLE table=test-table [id=7d6e9ae7afb04ac99e196c58139cd82f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:15:50.933393 18121 ts_tablet_manager.cc:1918] T 32a3094951154cd28eab69b16fbf002e P d9c57cc0f0484098bdf002dd0e84738b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:15:50.933472 18121 log.cc:1196] T 32a3094951154cd28eab69b16fbf002e P d9c57cc0f0484098bdf002dd0e84738b: Deleting WAL directory at /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/minicluster-data/ts-0/wal/wals/32a3094951154cd28eab69b16fbf002e
I20260809 02:15:50.933578 17535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d97e0d08daf642b3b123d9293bffba89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:15:50.933794 18121 ts_tablet_manager.cc:1939] T 32a3094951154cd28eab69b16fbf002e P d9c57cc0f0484098bdf002dd0e84738b: Deleting consensus metadata
I20260809 02:15:50.934921 17310 catalog_manager.cc:4841] TS d9c57cc0f0484098bdf002dd0e84738b (127.15.75.129:34125): tablet 32a3094951154cd28eab69b16fbf002e (table test-table [id=744891b29e0e444c9b2e7971359ead7c]) successfully deleted
I20260809 02:15:50.935685 18083 tablet_bootstrap.cc:492] T d97e0d08daf642b3b123d9293bffba89 P d9c57cc0f0484098bdf002dd0e84738b: Bootstrap starting.
I20260809 02:15:50.935890 17740 tablet_service.cc:1467] Processing CreateTablet for tablet d97e0d08daf642b3b123d9293bffba89 (DEFAULT_TABLE table=test-table [id=7d6e9ae7afb04ac99e196c58139cd82f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:15:50.936131 17740 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d97e0d08daf642b3b123d9293bffba89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:15:50.936728 18083 tablet_bootstrap.cc:654] T d97e0d08daf642b3b123d9293bffba89 P d9c57cc0f0484098bdf002dd0e84738b: Neither blocks nor log segments found. Creating new log.
I20260809 02:15:50.936967 17936 tablet_service.cc:1467] Processing CreateTablet for tablet d97e0d08daf642b3b123d9293bffba89 (DEFAULT_TABLE table=test-table [id=7d6e9ae7afb04ac99e196c58139cd82f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:15:50.937212 17936 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d97e0d08daf642b3b123d9293bffba89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:15:50.938252 18084 tablet_bootstrap.cc:492] T d97e0d08daf642b3b123d9293bffba89 P c209b3ffbd01442d9a920d60e1049c35: Bootstrap starting.
I20260809 02:15:50.939143 18084 tablet_bootstrap.cc:654] T d97e0d08daf642b3b123d9293bffba89 P c209b3ffbd01442d9a920d60e1049c35: Neither blocks nor log segments found. Creating new log.
I20260809 02:15:50.939538 18082 tablet_bootstrap.cc:492] T d97e0d08daf642b3b123d9293bffba89 P e96a2b7179e24ca5a6d70d700c420da6: Bootstrap starting.
I20260809 02:15:50.940062 18083 tablet_bootstrap.cc:492] T d97e0d08daf642b3b123d9293bffba89 P d9c57cc0f0484098bdf002dd0e84738b: No bootstrap required, opened a new log
I20260809 02:15:50.940152 18083 ts_tablet_manager.cc:1397] T d97e0d08daf642b3b123d9293bffba89 P d9c57cc0f0484098bdf002dd0e84738b: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260809 02:15:50.940542 18082 tablet_bootstrap.cc:654] T d97e0d08daf642b3b123d9293bffba89 P e96a2b7179e24ca5a6d70d700c420da6: Neither blocks nor log segments found. Creating new log.
I20260809 02:15:50.940656 18083 raft_consensus.cc:348] T d97e0d08daf642b3b123d9293bffba89 P d9c57cc0f0484098bdf002dd0e84738b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9c57cc0f0484098bdf002dd0e84738b" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 34125 } } peers { permanent_uuid: "c209b3ffbd01442d9a920d60e1049c35" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45793 } } peers { permanent_uuid: "e96a2b7179e24ca5a6d70d700c420da6" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 42807 } }
I20260809 02:15:50.940807 18083 raft_consensus.cc:374] T d97e0d08daf642b3b123d9293bffba89 P d9c57cc0f0484098bdf002dd0e84738b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:15:50.940848 18083 raft_consensus.cc:729] T d97e0d08daf642b3b123d9293bffba89 P d9c57cc0f0484098bdf002dd0e84738b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9c57cc0f0484098bdf002dd0e84738b, State: Initialized, Role: FOLLOWER
I20260809 02:15:50.940989 18083 consensus_queue.cc:260] T d97e0d08daf642b3b123d9293bffba89 P d9c57cc0f0484098bdf002dd0e84738b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9c57cc0f0484098bdf002dd0e84738b" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 34125 } } peers { permanent_uuid: "c209b3ffbd01442d9a920d60e1049c35" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45793 } } peers { permanent_uuid: "e96a2b7179e24ca5a6d70d700c420da6" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 42807 } }
I20260809 02:15:50.941293 18083 ts_tablet_manager.cc:1428] T d97e0d08daf642b3b123d9293bffba89 P d9c57cc0f0484098bdf002dd0e84738b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 02:15:50.942039 18122 ts_tablet_manager.cc:1905] T 32a3094951154cd28eab69b16fbf002e P e96a2b7179e24ca5a6d70d700c420da6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:15:50.942075 18084 tablet_bootstrap.cc:492] T d97e0d08daf642b3b123d9293bffba89 P c209b3ffbd01442d9a920d60e1049c35: No bootstrap required, opened a new log
I20260809 02:15:50.942166 18084 ts_tablet_manager.cc:1397] T d97e0d08daf642b3b123d9293bffba89 P c209b3ffbd01442d9a920d60e1049c35: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260809 02:15:50.942708 18084 raft_consensus.cc:348] T d97e0d08daf642b3b123d9293bffba89 P c209b3ffbd01442d9a920d60e1049c35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9c57cc0f0484098bdf002dd0e84738b" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 34125 } } peers { permanent_uuid: "c209b3ffbd01442d9a920d60e1049c35" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45793 } } peers { permanent_uuid: "e96a2b7179e24ca5a6d70d700c420da6" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 42807 } }
I20260809 02:15:50.942813 18084 raft_consensus.cc:374] T d97e0d08daf642b3b123d9293bffba89 P c209b3ffbd01442d9a920d60e1049c35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:15:50.942847 18084 raft_consensus.cc:729] T d97e0d08daf642b3b123d9293bffba89 P c209b3ffbd01442d9a920d60e1049c35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c209b3ffbd01442d9a920d60e1049c35, State: Initialized, Role: FOLLOWER
I20260809 02:15:50.942965 18084 consensus_queue.cc:260] T d97e0d08daf642b3b123d9293bffba89 P c209b3ffbd01442d9a920d60e1049c35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9c57cc0f0484098bdf002dd0e84738b" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 34125 } } peers { permanent_uuid: "c209b3ffbd01442d9a920d60e1049c35" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45793 } } peers { permanent_uuid: "e96a2b7179e24ca5a6d70d700c420da6" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 42807 } }
I20260809 02:15:50.943246 18084 ts_tablet_manager.cc:1428] T d97e0d08daf642b3b123d9293bffba89 P c209b3ffbd01442d9a920d60e1049c35: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 02:15:50.945473 18122 ts_tablet_manager.cc:1918] T 32a3094951154cd28eab69b16fbf002e P e96a2b7179e24ca5a6d70d700c420da6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:15:50.945588 18082 tablet_bootstrap.cc:492] T d97e0d08daf642b3b123d9293bffba89 P e96a2b7179e24ca5a6d70d700c420da6: No bootstrap required, opened a new log
I20260809 02:15:50.945613 18122 log.cc:1196] T 32a3094951154cd28eab69b16fbf002e P e96a2b7179e24ca5a6d70d700c420da6: Deleting WAL directory at /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/minicluster-data/ts-2/wal/wals/32a3094951154cd28eab69b16fbf002e
I20260809 02:15:50.945696 18082 ts_tablet_manager.cc:1397] T d97e0d08daf642b3b123d9293bffba89 P e96a2b7179e24ca5a6d70d700c420da6: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.002s
I20260809 02:15:50.946012 18122 ts_tablet_manager.cc:1939] T 32a3094951154cd28eab69b16fbf002e P e96a2b7179e24ca5a6d70d700c420da6: Deleting consensus metadata
I20260809 02:15:50.946125 18082 raft_consensus.cc:348] T d97e0d08daf642b3b123d9293bffba89 P e96a2b7179e24ca5a6d70d700c420da6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9c57cc0f0484098bdf002dd0e84738b" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 34125 } } peers { permanent_uuid: "c209b3ffbd01442d9a920d60e1049c35" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45793 } } peers { permanent_uuid: "e96a2b7179e24ca5a6d70d700c420da6" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 42807 } }
I20260809 02:15:50.946208 18082 raft_consensus.cc:374] T d97e0d08daf642b3b123d9293bffba89 P e96a2b7179e24ca5a6d70d700c420da6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:15:50.946228 18082 raft_consensus.cc:729] T d97e0d08daf642b3b123d9293bffba89 P e96a2b7179e24ca5a6d70d700c420da6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e96a2b7179e24ca5a6d70d700c420da6, State: Initialized, Role: FOLLOWER
I20260809 02:15:50.946363 18082 consensus_queue.cc:260] T d97e0d08daf642b3b123d9293bffba89 P e96a2b7179e24ca5a6d70d700c420da6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9c57cc0f0484098bdf002dd0e84738b" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 34125 } } peers { permanent_uuid: "c209b3ffbd01442d9a920d60e1049c35" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45793 } } peers { permanent_uuid: "e96a2b7179e24ca5a6d70d700c420da6" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 42807 } }
I20260809 02:15:50.946655 18082 ts_tablet_manager.cc:1428] T d97e0d08daf642b3b123d9293bffba89 P e96a2b7179e24ca5a6d70d700c420da6: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260809 02:15:50.947459 17314 catalog_manager.cc:4841] TS e96a2b7179e24ca5a6d70d700c420da6 (127.15.75.131:42807): tablet 32a3094951154cd28eab69b16fbf002e (table test-table [id=744891b29e0e444c9b2e7971359ead7c]) successfully deleted
W20260809 02:15:51.166414 17842 tablet.cc:2369] T d97e0d08daf642b3b123d9293bffba89 P c209b3ffbd01442d9a920d60e1049c35: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 02:15:51.186580 17635 tablet.cc:2369] T d97e0d08daf642b3b123d9293bffba89 P d9c57cc0f0484098bdf002dd0e84738b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:15:51.204946 18114 raft_consensus.cc:482] T d97e0d08daf642b3b123d9293bffba89 P c209b3ffbd01442d9a920d60e1049c35 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:15:51.205103 18114 raft_consensus.cc:504] T d97e0d08daf642b3b123d9293bffba89 P c209b3ffbd01442d9a920d60e1049c35 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9c57cc0f0484098bdf002dd0e84738b" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 34125 } } peers { permanent_uuid: "c209b3ffbd01442d9a920d60e1049c35" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45793 } } peers { permanent_uuid: "e96a2b7179e24ca5a6d70d700c420da6" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 42807 } }
I20260809 02:15:51.205610 18114 leader_election.cc:288] T d97e0d08daf642b3b123d9293bffba89 P c209b3ffbd01442d9a920d60e1049c35 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d9c57cc0f0484098bdf002dd0e84738b (127.15.75.129:34125), e96a2b7179e24ca5a6d70d700c420da6 (127.15.75.131:42807)
I20260809 02:15:51.206096 17971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d97e0d08daf642b3b123d9293bffba89" candidate_uuid: "c209b3ffbd01442d9a920d60e1049c35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e96a2b7179e24ca5a6d70d700c420da6" is_pre_election: true
I20260809 02:15:51.206136 17565 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d97e0d08daf642b3b123d9293bffba89" candidate_uuid: "c209b3ffbd01442d9a920d60e1049c35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9c57cc0f0484098bdf002dd0e84738b" is_pre_election: true
I20260809 02:15:51.206261 17971 raft_consensus.cc:2454] T d97e0d08daf642b3b123d9293bffba89 P e96a2b7179e24ca5a6d70d700c420da6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c209b3ffbd01442d9a920d60e1049c35 in term 0.
I20260809 02:15:51.206300 17565 raft_consensus.cc:2454] T d97e0d08daf642b3b123d9293bffba89 P d9c57cc0f0484098bdf002dd0e84738b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c209b3ffbd01442d9a920d60e1049c35 in term 0.
I20260809 02:15:51.206598 17677 leader_election.cc:302] T d97e0d08daf642b3b123d9293bffba89 P c209b3ffbd01442d9a920d60e1049c35 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c209b3ffbd01442d9a920d60e1049c35, e96a2b7179e24ca5a6d70d700c420da6; no voters:
I20260809 02:15:51.206794 18114 raft_consensus.cc:2781] T d97e0d08daf642b3b123d9293bffba89 P c209b3ffbd01442d9a920d60e1049c35 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:15:51.206864 18114 raft_consensus.cc:482] T d97e0d08daf642b3b123d9293bffba89 P c209b3ffbd01442d9a920d60e1049c35 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:15:51.206923 18114 raft_consensus.cc:3037] T d97e0d08daf642b3b123d9293bffba89 P c209b3ffbd01442d9a920d60e1049c35 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:15:51.207953 18114 raft_consensus.cc:504] T d97e0d08daf642b3b123d9293bffba89 P c209b3ffbd01442d9a920d60e1049c35 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9c57cc0f0484098bdf002dd0e84738b" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 34125 } } peers { permanent_uuid: "c209b3ffbd01442d9a920d60e1049c35" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45793 } } peers { permanent_uuid: "e96a2b7179e24ca5a6d70d700c420da6" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 42807 } }
I20260809 02:15:51.208323 18114 leader_election.cc:288] T d97e0d08daf642b3b123d9293bffba89 P c209b3ffbd01442d9a920d60e1049c35 [CANDIDATE]: Term 1 election: Requested vote from peers d9c57cc0f0484098bdf002dd0e84738b (127.15.75.129:34125), e96a2b7179e24ca5a6d70d700c420da6 (127.15.75.131:42807)
I20260809 02:15:51.208648 17565 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d97e0d08daf642b3b123d9293bffba89" candidate_uuid: "c209b3ffbd01442d9a920d60e1049c35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9c57cc0f0484098bdf002dd0e84738b"
I20260809 02:15:51.208739 17971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d97e0d08daf642b3b123d9293bffba89" candidate_uuid: "c209b3ffbd01442d9a920d60e1049c35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e96a2b7179e24ca5a6d70d700c420da6"
I20260809 02:15:51.208789 17565 raft_consensus.cc:3037] T d97e0d08daf642b3b123d9293bffba89 P d9c57cc0f0484098bdf002dd0e84738b [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:15:51.208858 17971 raft_consensus.cc:3037] T d97e0d08daf642b3b123d9293bffba89 P e96a2b7179e24ca5a6d70d700c420da6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:15:51.209944 17971 raft_consensus.cc:2454] T d97e0d08daf642b3b123d9293bffba89 P e96a2b7179e24ca5a6d70d700c420da6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c209b3ffbd01442d9a920d60e1049c35 in term 1.
I20260809 02:15:51.209944 17565 raft_consensus.cc:2454] T d97e0d08daf642b3b123d9293bffba89 P d9c57cc0f0484098bdf002dd0e84738b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c209b3ffbd01442d9a920d60e1049c35 in term 1.
I20260809 02:15:51.210253 17670 leader_election.cc:302] T d97e0d08daf642b3b123d9293bffba89 P c209b3ffbd01442d9a920d60e1049c35 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c209b3ffbd01442d9a920d60e1049c35, d9c57cc0f0484098bdf002dd0e84738b; no voters:
I20260809 02:15:51.210464 18114 raft_consensus.cc:2781] T d97e0d08daf642b3b123d9293bffba89 P c209b3ffbd01442d9a920d60e1049c35 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:15:51.210557 18114 raft_consensus.cc:686] T d97e0d08daf642b3b123d9293bffba89 P c209b3ffbd01442d9a920d60e1049c35 [term 1 LEADER]: Becoming Leader. State: Replica: c209b3ffbd01442d9a920d60e1049c35, State: Running, Role: LEADER
I20260809 02:15:51.210709 18114 consensus_queue.cc:237] T d97e0d08daf642b3b123d9293bffba89 P c209b3ffbd01442d9a920d60e1049c35 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9c57cc0f0484098bdf002dd0e84738b" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 34125 } } peers { permanent_uuid: "c209b3ffbd01442d9a920d60e1049c35" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45793 } } peers { permanent_uuid: "e96a2b7179e24ca5a6d70d700c420da6" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 42807 } }
I20260809 02:15:51.212387 17328 catalog_manager.cc:5495] T d97e0d08daf642b3b123d9293bffba89 P c209b3ffbd01442d9a920d60e1049c35 reported cstate change: term changed from 0 to 1, leader changed from <none> to c209b3ffbd01442d9a920d60e1049c35 (127.15.75.130). New cstate: current_term: 1 leader_uuid: "c209b3ffbd01442d9a920d60e1049c35" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9c57cc0f0484098bdf002dd0e84738b" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 34125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c209b3ffbd01442d9a920d60e1049c35" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45793 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e96a2b7179e24ca5a6d70d700c420da6" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 42807 } health_report { overall_health: UNKNOWN } } }
Aug 09 02:15:51 dist-test-slave-s11t krb5kdc[17266](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241749, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.190@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:15:51.258177 18138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:51.246212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:41691 (local address 127.0.0.1:53972)
0809 02:15:51.246489 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:51.246512 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:51.246595 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:51.246832 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:51.246834 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:51.247202 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:51.247310 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:51.247316 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:51.248041 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:51.248043 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:51.248601 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:51.248608 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:51.249100 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:51.249103 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:51.249167 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:51.253660 (+ 4493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:51.253672 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:51.255196 (+ 1524us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:15:51.256784 (+ 1588us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:15:51.256792 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:15:51.256797 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:15:51.257026 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:15:51.257275 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:15:51.257277 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:15:51.257278 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:15:51.257356 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:15:51.257696 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:15:51.257701 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:15:51.257799 (+ 98us) client_negotiation.cc:769] Sending connection context
0809 02:15:51.257907 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 02:15:51.258028 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":81,"threads_started":1}
I20260809 02:15:51.258215 18139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:51.246303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53972 (local address 127.15.75.190:41691)
0809 02:15:51.246561 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:51.246569 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:51.246582 (+ 13us) server_negotiation.cc:407] Connection header received
0809 02:15:51.246674 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:51.246676 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:51.246720 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:51.246794 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:51.247409 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:51.247946 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:51.248747 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:51.249000 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:51.249079 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:51.255335 (+ 6256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:15:51.255338 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:15:51.255340 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:15:51.255361 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:15:51.256630 (+ 1269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:51.257111 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:51.257114 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:51.257115 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:51.257193 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:51.257433 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:51.257436 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:51.257437 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:51.257606 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:15:51.257683 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:51.257965 (+ 282us) server_negotiation.cc:299] Negotiation successful
0809 02:15:51.258087 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":16,"server-negotiator.queue_time_us":182,"thread_start_us":107,"threads_started":1}
I20260809 02:15:51.262046 17328 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.15.75.190@KRBTEST.COM'} at 127.0.0.1:53972:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 02:15:51.262830 17328 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:15:51.267853 17535 tablet_service.cc:1467] Processing CreateTablet for tablet 599d9f6b047044f48b2647e90727a567 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=aeecc41f7f3c487e97bb0ed9beb9d26e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 02:15:51.268024 17936 tablet_service.cc:1467] Processing CreateTablet for tablet 599d9f6b047044f48b2647e90727a567 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=aeecc41f7f3c487e97bb0ed9beb9d26e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 02:15:51.268289 17535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 599d9f6b047044f48b2647e90727a567. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:15:51.268492 17936 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 599d9f6b047044f48b2647e90727a567. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:15:51.267853 17740 tablet_service.cc:1467] Processing CreateTablet for tablet 599d9f6b047044f48b2647e90727a567 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=aeecc41f7f3c487e97bb0ed9beb9d26e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 02:15:51.270411 17740 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 599d9f6b047044f48b2647e90727a567. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:15:51.270828 18083 tablet_bootstrap.cc:492] T 599d9f6b047044f48b2647e90727a567 P d9c57cc0f0484098bdf002dd0e84738b: Bootstrap starting.
I20260809 02:15:51.271080 18082 tablet_bootstrap.cc:492] T 599d9f6b047044f48b2647e90727a567 P e96a2b7179e24ca5a6d70d700c420da6: Bootstrap starting.
I20260809 02:15:51.271811 18083 tablet_bootstrap.cc:654] T 599d9f6b047044f48b2647e90727a567 P d9c57cc0f0484098bdf002dd0e84738b: Neither blocks nor log segments found. Creating new log.
I20260809 02:15:51.271945 18082 tablet_bootstrap.cc:654] T 599d9f6b047044f48b2647e90727a567 P e96a2b7179e24ca5a6d70d700c420da6: Neither blocks nor log segments found. Creating new log.
I20260809 02:15:51.273320 18082 tablet_bootstrap.cc:492] T 599d9f6b047044f48b2647e90727a567 P e96a2b7179e24ca5a6d70d700c420da6: No bootstrap required, opened a new log
I20260809 02:15:51.273320 18083 tablet_bootstrap.cc:492] T 599d9f6b047044f48b2647e90727a567 P d9c57cc0f0484098bdf002dd0e84738b: No bootstrap required, opened a new log
I20260809 02:15:51.273408 18083 ts_tablet_manager.cc:1397] T 599d9f6b047044f48b2647e90727a567 P d9c57cc0f0484098bdf002dd0e84738b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 02:15:51.273408 18082 ts_tablet_manager.cc:1397] T 599d9f6b047044f48b2647e90727a567 P e96a2b7179e24ca5a6d70d700c420da6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260809 02:15:51.273417 18084 tablet_bootstrap.cc:492] T 599d9f6b047044f48b2647e90727a567 P c209b3ffbd01442d9a920d60e1049c35: Bootstrap starting.
I20260809 02:15:51.273855 18082 raft_consensus.cc:348] T 599d9f6b047044f48b2647e90727a567 P e96a2b7179e24ca5a6d70d700c420da6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9c57cc0f0484098bdf002dd0e84738b" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 34125 } } peers { permanent_uuid: "c209b3ffbd01442d9a920d60e1049c35" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45793 } } peers { permanent_uuid: "e96a2b7179e24ca5a6d70d700c420da6" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 42807 } }
I20260809 02:15:51.273844 18083 raft_consensus.cc:348] T 599d9f6b047044f48b2647e90727a567 P d9c57cc0f0484098bdf002dd0e84738b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9c57cc0f0484098bdf002dd0e84738b" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 34125 } } peers { permanent_uuid: "c209b3ffbd01442d9a920d60e1049c35" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45793 } } peers { permanent_uuid: "e96a2b7179e24ca5a6d70d700c420da6" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 42807 } }
I20260809 02:15:51.273954 18082 raft_consensus.cc:374] T 599d9f6b047044f48b2647e90727a567 P e96a2b7179e24ca5a6d70d700c420da6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:15:51.273954 18083 raft_consensus.cc:374] T 599d9f6b047044f48b2647e90727a567 P d9c57cc0f0484098bdf002dd0e84738b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:15:51.273988 18082 raft_consensus.cc:729] T 599d9f6b047044f48b2647e90727a567 P e96a2b7179e24ca5a6d70d700c420da6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e96a2b7179e24ca5a6d70d700c420da6, State: Initialized, Role: FOLLOWER
I20260809 02:15:51.273988 18083 raft_consensus.cc:729] T 599d9f6b047044f48b2647e90727a567 P d9c57cc0f0484098bdf002dd0e84738b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9c57cc0f0484098bdf002dd0e84738b, State: Initialized, Role: FOLLOWER
I20260809 02:15:51.274117 18083 consensus_queue.cc:260] T 599d9f6b047044f48b2647e90727a567 P d9c57cc0f0484098bdf002dd0e84738b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9c57cc0f0484098bdf002dd0e84738b" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 34125 } } peers { permanent_uuid: "c209b3ffbd01442d9a920d60e1049c35" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45793 } } peers { permanent_uuid: "e96a2b7179e24ca5a6d70d700c420da6" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 42807 } }
I20260809 02:15:51.274117 18082 consensus_queue.cc:260] T 599d9f6b047044f48b2647e90727a567 P e96a2b7179e24ca5a6d70d700c420da6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9c57cc0f0484098bdf002dd0e84738b" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 34125 } } peers { permanent_uuid: "c209b3ffbd01442d9a920d60e1049c35" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45793 } } peers { permanent_uuid: "e96a2b7179e24ca5a6d70d700c420da6" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 42807 } }
I20260809 02:15:51.274266 18084 tablet_bootstrap.cc:654] T 599d9f6b047044f48b2647e90727a567 P c209b3ffbd01442d9a920d60e1049c35: Neither blocks nor log segments found. Creating new log.
I20260809 02:15:51.274403 18083 ts_tablet_manager.cc:1428] T 599d9f6b047044f48b2647e90727a567 P d9c57cc0f0484098bdf002dd0e84738b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 02:15:51.274397 18082 ts_tablet_manager.cc:1428] T 599d9f6b047044f48b2647e90727a567 P e96a2b7179e24ca5a6d70d700c420da6: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 02:15:51.274521 18091 tablet_replica.cc:439] T 599d9f6b047044f48b2647e90727a567 P d9c57cc0f0484098bdf002dd0e84738b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9c57cc0f0484098bdf002dd0e84738b" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 34125 } } peers { permanent_uuid: "c209b3ffbd01442d9a920d60e1049c35" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45793 } } peers { permanent_uuid: "e96a2b7179e24ca5a6d70d700c420da6" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 42807 } } }
I20260809 02:15:51.274657 18091 tablet_replica.cc:442] T 599d9f6b047044f48b2647e90727a567 P d9c57cc0f0484098bdf002dd0e84738b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 02:15:51.275045 18092 tablet_replica.cc:439] T 599d9f6b047044f48b2647e90727a567 P e96a2b7179e24ca5a6d70d700c420da6: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9c57cc0f0484098bdf002dd0e84738b" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 34125 } } peers { permanent_uuid: "c209b3ffbd01442d9a920d60e1049c35" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45793 } } peers { permanent_uuid: "e96a2b7179e24ca5a6d70d700c420da6" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 42807 } } }
I20260809 02:15:51.275211 18092 tablet_replica.cc:442] T 599d9f6b047044f48b2647e90727a567 P e96a2b7179e24ca5a6d70d700c420da6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 02:15:51.275776 18084 tablet_bootstrap.cc:492] T 599d9f6b047044f48b2647e90727a567 P c209b3ffbd01442d9a920d60e1049c35: No bootstrap required, opened a new log
I20260809 02:15:51.275861 18084 ts_tablet_manager.cc:1397] T 599d9f6b047044f48b2647e90727a567 P c209b3ffbd01442d9a920d60e1049c35: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260809 02:15:51.276301 18084 raft_consensus.cc:348] T 599d9f6b047044f48b2647e90727a567 P c209b3ffbd01442d9a920d60e1049c35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9c57cc0f0484098bdf002dd0e84738b" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 34125 } } peers { permanent_uuid: "c209b3ffbd01442d9a920d60e1049c35" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45793 } } peers { permanent_uuid: "e96a2b7179e24ca5a6d70d700c420da6" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 42807 } }
I20260809 02:15:51.276412 18084 raft_consensus.cc:374] T 599d9f6b047044f48b2647e90727a567 P c209b3ffbd01442d9a920d60e1049c35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:15:51.276450 18084 raft_consensus.cc:729] T 599d9f6b047044f48b2647e90727a567 P c209b3ffbd01442d9a920d60e1049c35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c209b3ffbd01442d9a920d60e1049c35, State: Initialized, Role: FOLLOWER
I20260809 02:15:51.276575 18084 consensus_queue.cc:260] T 599d9f6b047044f48b2647e90727a567 P c209b3ffbd01442d9a920d60e1049c35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9c57cc0f0484098bdf002dd0e84738b" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 34125 } } peers { permanent_uuid: "c209b3ffbd01442d9a920d60e1049c35" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45793 } } peers { permanent_uuid: "e96a2b7179e24ca5a6d70d700c420da6" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 42807 } }
I20260809 02:15:51.276844 18114 tablet_replica.cc:439] T 599d9f6b047044f48b2647e90727a567 P c209b3ffbd01442d9a920d60e1049c35: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9c57cc0f0484098bdf002dd0e84738b" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 34125 } } peers { permanent_uuid: "c209b3ffbd01442d9a920d60e1049c35" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45793 } } peers { permanent_uuid: "e96a2b7179e24ca5a6d70d700c420da6" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 42807 } } }
I20260809 02:15:51.276984 18114 tablet_replica.cc:442] T 599d9f6b047044f48b2647e90727a567 P c209b3ffbd01442d9a920d60e1049c35: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 02:15:51.276850 18084 ts_tablet_manager.cc:1428] T 599d9f6b047044f48b2647e90727a567 P c209b3ffbd01442d9a920d60e1049c35: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 02:15:51.329249 18091 raft_consensus.cc:482] T 599d9f6b047044f48b2647e90727a567 P d9c57cc0f0484098bdf002dd0e84738b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:15:51.329372 18091 raft_consensus.cc:504] T 599d9f6b047044f48b2647e90727a567 P d9c57cc0f0484098bdf002dd0e84738b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9c57cc0f0484098bdf002dd0e84738b" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 34125 } } peers { permanent_uuid: "c209b3ffbd01442d9a920d60e1049c35" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45793 } } peers { permanent_uuid: "e96a2b7179e24ca5a6d70d700c420da6" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 42807 } }
I20260809 02:15:51.330394 18091 leader_election.cc:288] T 599d9f6b047044f48b2647e90727a567 P d9c57cc0f0484098bdf002dd0e84738b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c209b3ffbd01442d9a920d60e1049c35 (127.15.75.130:45793), e96a2b7179e24ca5a6d70d700c420da6 (127.15.75.131:42807)
I20260809 02:15:51.334683 18074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:51.330752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.129:50243 (local address 127.15.75.131:42807)
0809 02:15:51.330914 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:51.330918 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:51.331149 (+ 231us) server_negotiation.cc:407] Connection header received
0809 02:15:51.331299 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:51.331302 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:51.331354 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:51.331435 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:15:51.331790 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:51.332292 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:51.333574 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:51.334093 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:51.334189 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:51.334228 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:51.334472 (+ 244us) server_negotiation.cc:299] Negotiation successful
0809 02:15:51.334561 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 02:15:51.334697 18144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:51.330631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.131:42807 (local address 127.15.75.129:50243)
0809 02:15:51.331055 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:51.331068 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:51.331200 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:51.331459 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:51.331462 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:51.331519 (+ 57us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:15:51.331677 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:51.331683 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:51.332405 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:51.332410 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:51.333441 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:51.333450 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:51.334214 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:51.334217 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:51.334336 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:51.334364 (+ 28us) client_negotiation.cc:769] Sending connection context
0809 02:15:51.334440 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 02:15:51.334513 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"spinlock_wait_cycles":20352,"thread_start_us":136,"threads_started":1}
I20260809 02:15:51.334802 18081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:51.330731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.129:46793 (local address 127.15.75.130:45793)
0809 02:15:51.330869 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:51.330874 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:51.331203 (+ 329us) server_negotiation.cc:407] Connection header received
0809 02:15:51.331349 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:51.331352 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:51.331409 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:51.331494 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:15:51.331807 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:51.332392 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:51.333728 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:51.334225 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:51.334355 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:51.334415 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:51.334599 (+ 184us) server_negotiation.cc:299] Negotiation successful
0809 02:15:51.334674 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 02:15:51.334919 18145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:51.330631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.130:45793 (local address 127.15.75.129:46793)
0809 02:15:51.331129 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:51.331141 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:51.331227 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:51.331542 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:51.331546 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:51.331587 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:15:51.331693 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:51.331698 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:51.332529 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:51.332539 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:51.333605 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:51.333616 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:51.334352 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:51.334355 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:51.334465 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:51.334491 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 02:15:51.334565 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 02:15:51.334628 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":420,"mutex_wait_us":180,"spinlock_wait_cycles":768,"thread_start_us":103,"threads_started":1}
I20260809 02:15:51.335253 17971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "599d9f6b047044f48b2647e90727a567" candidate_uuid: "d9c57cc0f0484098bdf002dd0e84738b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e96a2b7179e24ca5a6d70d700c420da6" is_pre_election: true
I20260809 02:15:51.335414 17971 raft_consensus.cc:2454] T 599d9f6b047044f48b2647e90727a567 P e96a2b7179e24ca5a6d70d700c420da6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9c57cc0f0484098bdf002dd0e84738b in term 0.
I20260809 02:15:51.335492 17772 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "599d9f6b047044f48b2647e90727a567" candidate_uuid: "d9c57cc0f0484098bdf002dd0e84738b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c209b3ffbd01442d9a920d60e1049c35" is_pre_election: true
I20260809 02:15:51.335744 17772 raft_consensus.cc:2454] T 599d9f6b047044f48b2647e90727a567 P c209b3ffbd01442d9a920d60e1049c35 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9c57cc0f0484098bdf002dd0e84738b in term 0.
I20260809 02:15:51.335803 17461 leader_election.cc:302] T 599d9f6b047044f48b2647e90727a567 P d9c57cc0f0484098bdf002dd0e84738b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d9c57cc0f0484098bdf002dd0e84738b, e96a2b7179e24ca5a6d70d700c420da6; no voters:
I20260809 02:15:51.336061 18091 raft_consensus.cc:2781] T 599d9f6b047044f48b2647e90727a567 P d9c57cc0f0484098bdf002dd0e84738b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:15:51.336149 18091 raft_consensus.cc:482] T 599d9f6b047044f48b2647e90727a567 P d9c57cc0f0484098bdf002dd0e84738b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:15:51.336191 18091 raft_consensus.cc:3037] T 599d9f6b047044f48b2647e90727a567 P d9c57cc0f0484098bdf002dd0e84738b [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:15:51.337600 18091 raft_consensus.cc:504] T 599d9f6b047044f48b2647e90727a567 P d9c57cc0f0484098bdf002dd0e84738b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9c57cc0f0484098bdf002dd0e84738b" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 34125 } } peers { permanent_uuid: "c209b3ffbd01442d9a920d60e1049c35" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45793 } } peers { permanent_uuid: "e96a2b7179e24ca5a6d70d700c420da6" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 42807 } }
I20260809 02:15:51.337913 18091 leader_election.cc:288] T 599d9f6b047044f48b2647e90727a567 P d9c57cc0f0484098bdf002dd0e84738b [CANDIDATE]: Term 1 election: Requested vote from peers c209b3ffbd01442d9a920d60e1049c35 (127.15.75.130:45793), e96a2b7179e24ca5a6d70d700c420da6 (127.15.75.131:42807)
I20260809 02:15:51.338181 17772 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "599d9f6b047044f48b2647e90727a567" candidate_uuid: "d9c57cc0f0484098bdf002dd0e84738b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c209b3ffbd01442d9a920d60e1049c35"
I20260809 02:15:51.338313 17772 raft_consensus.cc:3037] T 599d9f6b047044f48b2647e90727a567 P c209b3ffbd01442d9a920d60e1049c35 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:15:51.338277 17971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "599d9f6b047044f48b2647e90727a567" candidate_uuid: "d9c57cc0f0484098bdf002dd0e84738b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e96a2b7179e24ca5a6d70d700c420da6"
I20260809 02:15:51.338404 17971 raft_consensus.cc:3037] T 599d9f6b047044f48b2647e90727a567 P e96a2b7179e24ca5a6d70d700c420da6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:15:51.339239 17772 raft_consensus.cc:2454] T 599d9f6b047044f48b2647e90727a567 P c209b3ffbd01442d9a920d60e1049c35 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9c57cc0f0484098bdf002dd0e84738b in term 1.
I20260809 02:15:51.339458 17971 raft_consensus.cc:2454] T 599d9f6b047044f48b2647e90727a567 P e96a2b7179e24ca5a6d70d700c420da6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9c57cc0f0484098bdf002dd0e84738b in term 1.
I20260809 02:15:51.339543 17460 leader_election.cc:302] T 599d9f6b047044f48b2647e90727a567 P d9c57cc0f0484098bdf002dd0e84738b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c209b3ffbd01442d9a920d60e1049c35, d9c57cc0f0484098bdf002dd0e84738b; no voters:
I20260809 02:15:51.339710 18091 raft_consensus.cc:2781] T 599d9f6b047044f48b2647e90727a567 P d9c57cc0f0484098bdf002dd0e84738b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:15:51.339926 18091 raft_consensus.cc:686] T 599d9f6b047044f48b2647e90727a567 P d9c57cc0f0484098bdf002dd0e84738b [term 1 LEADER]: Becoming Leader. State: Replica: d9c57cc0f0484098bdf002dd0e84738b, State: Running, Role: LEADER
I20260809 02:15:51.340166 18091 consensus_queue.cc:237] T 599d9f6b047044f48b2647e90727a567 P d9c57cc0f0484098bdf002dd0e84738b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9c57cc0f0484098bdf002dd0e84738b" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 34125 } } peers { permanent_uuid: "c209b3ffbd01442d9a920d60e1049c35" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45793 } } peers { permanent_uuid: "e96a2b7179e24ca5a6d70d700c420da6" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 42807 } }
I20260809 02:15:51.341531 18146 tablet_replica.cc:439] T 599d9f6b047044f48b2647e90727a567 P d9c57cc0f0484098bdf002dd0e84738b: TxnStatusTablet state changed. Reason: New leader d9c57cc0f0484098bdf002dd0e84738b. Latest consensus state: current_term: 1 leader_uuid: "d9c57cc0f0484098bdf002dd0e84738b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9c57cc0f0484098bdf002dd0e84738b" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 34125 } } peers { permanent_uuid: "c209b3ffbd01442d9a920d60e1049c35" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45793 } } peers { permanent_uuid: "e96a2b7179e24ca5a6d70d700c420da6" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 42807 } } }
I20260809 02:15:51.341643 18146 tablet_replica.cc:442] T 599d9f6b047044f48b2647e90727a567 P d9c57cc0f0484098bdf002dd0e84738b: This TxnStatusTablet replica's current role is: LEADER
I20260809 02:15:51.341976 18148 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 02:15:51.342190 18148 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 02:15:51.342698 17326 catalog_manager.cc:5495] T 599d9f6b047044f48b2647e90727a567 P d9c57cc0f0484098bdf002dd0e84738b reported cstate change: term changed from 0 to 1, leader changed from <none> to d9c57cc0f0484098bdf002dd0e84738b (127.15.75.129). New cstate: current_term: 1 leader_uuid: "d9c57cc0f0484098bdf002dd0e84738b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9c57cc0f0484098bdf002dd0e84738b" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 34125 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c209b3ffbd01442d9a920d60e1049c35" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45793 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e96a2b7179e24ca5a6d70d700c420da6" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 42807 } health_report { overall_health: UNKNOWN } } }
I20260809 02:15:51.358958 18153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:51.354924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46768 (local address 127.15.75.129:34125)
0809 02:15:51.355236 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:51.355243 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:51.355254 (+ 11us) server_negotiation.cc:407] Connection header received
0809 02:15:51.355309 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:51.355311 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:51.355360 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:51.355451 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:15:51.355744 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:51.356301 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:51.357503 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:51.357741 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:51.357819 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:51.358156 (+ 337us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:15:51.358544 (+ 388us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:15:51.358626 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:51.358747 (+ 121us) server_negotiation.cc:299] Negotiation successful
0809 02:15:51.358793 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":231,"thread_start_us":172,"threads_started":1}
I20260809 02:15:51.358954 18138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:51.354845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.129:34125 (local address 127.0.0.1:46768)
0809 02:15:51.355017 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:51.355034 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:51.355133 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:51.355491 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:51.355495 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:51.355503 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 02:15:51.355621 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:51.355629 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:51.356404 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:51.356408 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:51.357390 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:51.357402 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:51.357846 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:51.357850 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:51.357958 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:51.358013 (+ 55us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 02:15:51.358637 (+ 624us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 02:15:51.358652 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 02:15:51.358728 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 02:15:51.358812 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260809 02:15:51.361552 17971 raft_consensus.cc:1261] T 599d9f6b047044f48b2647e90727a567 P e96a2b7179e24ca5a6d70d700c420da6 [term 1 FOLLOWER]: Refusing update from remote peer d9c57cc0f0484098bdf002dd0e84738b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:15:51.361714 17772 raft_consensus.cc:1261] T 599d9f6b047044f48b2647e90727a567 P c209b3ffbd01442d9a920d60e1049c35 [term 1 FOLLOWER]: Refusing update from remote peer d9c57cc0f0484098bdf002dd0e84738b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:15:51.362103 18146 consensus_queue.cc:1035] T 599d9f6b047044f48b2647e90727a567 P d9c57cc0f0484098bdf002dd0e84738b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e96a2b7179e24ca5a6d70d700c420da6" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 42807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:15:51.362257 18091 consensus_queue.cc:1035] T 599d9f6b047044f48b2647e90727a567 P d9c57cc0f0484098bdf002dd0e84738b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c209b3ffbd01442d9a920d60e1049c35" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:15:51.363698 18092 tablet_replica.cc:439] T 599d9f6b047044f48b2647e90727a567 P e96a2b7179e24ca5a6d70d700c420da6: TxnStatusTablet state changed. Reason: New leader d9c57cc0f0484098bdf002dd0e84738b. Latest consensus state: current_term: 1 leader_uuid: "d9c57cc0f0484098bdf002dd0e84738b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9c57cc0f0484098bdf002dd0e84738b" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 34125 } } peers { permanent_uuid: "c209b3ffbd01442d9a920d60e1049c35" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45793 } } peers { permanent_uuid: "e96a2b7179e24ca5a6d70d700c420da6" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 42807 } } }
I20260809 02:15:51.363804 18092 tablet_replica.cc:442] T 599d9f6b047044f48b2647e90727a567 P e96a2b7179e24ca5a6d70d700c420da6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 02:15:51.365804 18091 tablet_replica.cc:439] T 599d9f6b047044f48b2647e90727a567 P d9c57cc0f0484098bdf002dd0e84738b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d9c57cc0f0484098bdf002dd0e84738b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9c57cc0f0484098bdf002dd0e84738b" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 34125 } } peers { permanent_uuid: "c209b3ffbd01442d9a920d60e1049c35" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45793 } } peers { permanent_uuid: "e96a2b7179e24ca5a6d70d700c420da6" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 42807 } } }
I20260809 02:15:51.365916 18091 tablet_replica.cc:442] T 599d9f6b047044f48b2647e90727a567 P d9c57cc0f0484098bdf002dd0e84738b: This TxnStatusTablet replica's current role is: LEADER
I20260809 02:15:51.366930 18094 tablet_replica.cc:439] T 599d9f6b047044f48b2647e90727a567 P c209b3ffbd01442d9a920d60e1049c35: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d9c57cc0f0484098bdf002dd0e84738b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9c57cc0f0484098bdf002dd0e84738b" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 34125 } } peers { permanent_uuid: "c209b3ffbd01442d9a920d60e1049c35" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45793 } } peers { permanent_uuid: "e96a2b7179e24ca5a6d70d700c420da6" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 42807 } } }
I20260809 02:15:51.367041 18094 tablet_replica.cc:442] T 599d9f6b047044f48b2647e90727a567 P c209b3ffbd01442d9a920d60e1049c35: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 02:15:51.367244 18114 tablet_replica.cc:439] T 599d9f6b047044f48b2647e90727a567 P c209b3ffbd01442d9a920d60e1049c35: TxnStatusTablet state changed. Reason: New leader d9c57cc0f0484098bdf002dd0e84738b. Latest consensus state: current_term: 1 leader_uuid: "d9c57cc0f0484098bdf002dd0e84738b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9c57cc0f0484098bdf002dd0e84738b" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 34125 } } peers { permanent_uuid: "c209b3ffbd01442d9a920d60e1049c35" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45793 } } peers { permanent_uuid: "e96a2b7179e24ca5a6d70d700c420da6" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 42807 } } }
I20260809 02:15:51.367328 18114 tablet_replica.cc:442] T 599d9f6b047044f48b2647e90727a567 P c209b3ffbd01442d9a920d60e1049c35: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 02:15:51.370430 18146 tablet_replica.cc:439] T 599d9f6b047044f48b2647e90727a567 P d9c57cc0f0484098bdf002dd0e84738b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d9c57cc0f0484098bdf002dd0e84738b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9c57cc0f0484098bdf002dd0e84738b" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 34125 } } peers { permanent_uuid: "c209b3ffbd01442d9a920d60e1049c35" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45793 } } peers { permanent_uuid: "e96a2b7179e24ca5a6d70d700c420da6" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 42807 } } }
I20260809 02:15:51.370532 18146 tablet_replica.cc:442] T 599d9f6b047044f48b2647e90727a567 P d9c57cc0f0484098bdf002dd0e84738b: This TxnStatusTablet replica's current role is: LEADER
I20260809 02:15:51.371497 18092 tablet_replica.cc:439] T 599d9f6b047044f48b2647e90727a567 P e96a2b7179e24ca5a6d70d700c420da6: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d9c57cc0f0484098bdf002dd0e84738b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9c57cc0f0484098bdf002dd0e84738b" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 34125 } } peers { permanent_uuid: "c209b3ffbd01442d9a920d60e1049c35" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45793 } } peers { permanent_uuid: "e96a2b7179e24ca5a6d70d700c420da6" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 42807 } } }
I20260809 02:15:51.371600 18092 tablet_replica.cc:442] T 599d9f6b047044f48b2647e90727a567 P e96a2b7179e24ca5a6d70d700c420da6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 02:15:51.382489 18153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:51.379144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.130:34253 (local address 127.15.75.129:34125)
0809 02:15:51.379236 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:51.379239 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:51.379460 (+ 221us) server_negotiation.cc:407] Connection header received
0809 02:15:51.379617 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:51.379621 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:51.379665 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:51.379753 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:15:51.379994 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:51.380501 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:51.381427 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:51.381668 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:51.381793 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:51.382066 (+ 273us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:15:51.382141 (+ 75us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:15:51.382222 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:51.382311 (+ 89us) server_negotiation.cc:299] Negotiation successful
0809 02:15:51.382367 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 02:15:51.382527 18162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:51.379067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.129:34125 (local address 127.15.75.130:34253)
0809 02:15:51.379375 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:51.379387 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:51.379513 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:51.379767 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:51.379770 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:51.379778 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 02:15:51.379889 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:51.379894 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:51.380617 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:51.380620 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:51.381304 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:51.381313 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:51.381784 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:51.381788 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:51.381887 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:51.381929 (+ 42us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 02:15:51.382228 (+ 299us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 02:15:51.382238 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 02:15:51.382302 (+ 64us) client_negotiation.cc:240] Negotiation successful
0809 02:15:51.382389 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":75,"threads_started":1}
I20260809 02:15:51.389175 17971 raft_consensus.cc:1261] T d97e0d08daf642b3b123d9293bffba89 P e96a2b7179e24ca5a6d70d700c420da6 [term 1 FOLLOWER]: Refusing update from remote peer c209b3ffbd01442d9a920d60e1049c35: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:15:51.389415 17565 raft_consensus.cc:1261] T d97e0d08daf642b3b123d9293bffba89 P d9c57cc0f0484098bdf002dd0e84738b [term 1 FOLLOWER]: Refusing update from remote peer c209b3ffbd01442d9a920d60e1049c35: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:15:51.389585 18114 consensus_queue.cc:1035] T d97e0d08daf642b3b123d9293bffba89 P c209b3ffbd01442d9a920d60e1049c35 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e96a2b7179e24ca5a6d70d700c420da6" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 42807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:15:51.389811 18094 consensus_queue.cc:1035] T d97e0d08daf642b3b123d9293bffba89 P c209b3ffbd01442d9a920d60e1049c35 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9c57cc0f0484098bdf002dd0e84738b" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 34125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:15:51.412209 18081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:51.408248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.129:53449 (local address 127.15.75.130:45793)
0809 02:15:51.408395 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:51.408400 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:51.408596 (+ 196us) server_negotiation.cc:407] Connection header received
0809 02:15:51.408738 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:51.408741 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:51.408791 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:51.408855 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:15:51.409145 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:51.409650 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:51.410919 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:51.411261 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:51.411334 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:51.411739 (+ 405us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:15:51.411807 (+ 68us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:15:51.411920 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:51.412022 (+ 102us) server_negotiation.cc:299] Negotiation successful
0809 02:15:51.412071 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 02:15:51.412249 18167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:51.408189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.130:45793 (local address 127.15.75.129:53449)
0809 02:15:51.408507 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:51.408522 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:51.408626 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:51.408883 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:51.408886 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:51.408903 (+ 17us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 02:15:51.409030 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:51.409036 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:51.409750 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:51.409754 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:51.410770 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:51.410782 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:51.411463 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:51.411467 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:51.411562 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:51.411611 (+ 49us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 02:15:51.411923 (+ 312us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 02:15:51.411933 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 02:15:51.412001 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 02:15:51.412079 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":121,"threads_started":1}
I20260809 02:15:51.444131 17326 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53964:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:15:51.444303 17326 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53964:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:15:51.446466 17326 catalog_manager.cc:5782] T 00000000000000000000000000000000 P df9490defb084e519d100b3d99641ab0: Sending DeleteTablet for 3 replicas of tablet d97e0d08daf642b3b123d9293bffba89
I20260809 02:15:51.447039 17740 tablet_service.cc:1514] Processing DeleteTablet for tablet d97e0d08daf642b3b123d9293bffba89 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:15:51 UTC) from {username='kudu', principal='kudu/127.15.75.190@KRBTEST.COM'} at 127.0.0.1:49748
I20260809 02:15:51.447062 17936 tablet_service.cc:1514] Processing DeleteTablet for tablet d97e0d08daf642b3b123d9293bffba89 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:15:51 UTC) from {username='kudu', principal='kudu/127.15.75.190@KRBTEST.COM'} at 127.0.0.1:48552
I20260809 02:15:51.447304 18122 tablet_replica.cc:330] T d97e0d08daf642b3b123d9293bffba89 P e96a2b7179e24ca5a6d70d700c420da6: stopping tablet replica
I20260809 02:15:51.447340 17535 tablet_service.cc:1514] Processing DeleteTablet for tablet d97e0d08daf642b3b123d9293bffba89 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:15:51 UTC) from {username='kudu', principal='kudu/127.15.75.190@KRBTEST.COM'} at 127.0.0.1:46764
I20260809 02:15:51.447420 18122 raft_consensus.cc:2229] T d97e0d08daf642b3b123d9293bffba89 P e96a2b7179e24ca5a6d70d700c420da6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:15:51.447527 18122 raft_consensus.cc:2258] T d97e0d08daf642b3b123d9293bffba89 P e96a2b7179e24ca5a6d70d700c420da6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:15:51.447602 18171 tablet_replica.cc:330] T d97e0d08daf642b3b123d9293bffba89 P d9c57cc0f0484098bdf002dd0e84738b: stopping tablet replica
I20260809 02:15:51.447902 18171 raft_consensus.cc:2229] T d97e0d08daf642b3b123d9293bffba89 P d9c57cc0f0484098bdf002dd0e84738b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:15:51.447937 18169 tablet_replica.cc:330] T d97e0d08daf642b3b123d9293bffba89 P c209b3ffbd01442d9a920d60e1049c35: stopping tablet replica
I20260809 02:15:51.448009 18171 raft_consensus.cc:2258] T d97e0d08daf642b3b123d9293bffba89 P d9c57cc0f0484098bdf002dd0e84738b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:15:51.448150 18169 raft_consensus.cc:2229] T d97e0d08daf642b3b123d9293bffba89 P c209b3ffbd01442d9a920d60e1049c35 [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:15:51.448357 18169 raft_consensus.cc:2258] T d97e0d08daf642b3b123d9293bffba89 P c209b3ffbd01442d9a920d60e1049c35 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:15:51.448616 18122 ts_tablet_manager.cc:1905] T d97e0d08daf642b3b123d9293bffba89 P e96a2b7179e24ca5a6d70d700c420da6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:15:51.449111 18169 ts_tablet_manager.cc:1905] T d97e0d08daf642b3b123d9293bffba89 P c209b3ffbd01442d9a920d60e1049c35: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:15:51.451459 18122 ts_tablet_manager.cc:1918] T d97e0d08daf642b3b123d9293bffba89 P e96a2b7179e24ca5a6d70d700c420da6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 02:15:51.451540 18122 log.cc:1196] T d97e0d08daf642b3b123d9293bffba89 P e96a2b7179e24ca5a6d70d700c420da6: Deleting WAL directory at /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/minicluster-data/ts-2/wal/wals/d97e0d08daf642b3b123d9293bffba89
I20260809 02:15:51.451794 18122 ts_tablet_manager.cc:1939] T d97e0d08daf642b3b123d9293bffba89 P e96a2b7179e24ca5a6d70d700c420da6: Deleting consensus metadata
I20260809 02:15:51.452399 18169 ts_tablet_manager.cc:1918] T d97e0d08daf642b3b123d9293bffba89 P c209b3ffbd01442d9a920d60e1049c35: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 02:15:51.452418 17314 catalog_manager.cc:4841] TS e96a2b7179e24ca5a6d70d700c420da6 (127.15.75.131:42807): tablet d97e0d08daf642b3b123d9293bffba89 (table test-table [id=7d6e9ae7afb04ac99e196c58139cd82f]) successfully deleted
I20260809 02:15:51.452564 18169 log.cc:1196] T d97e0d08daf642b3b123d9293bffba89 P c209b3ffbd01442d9a920d60e1049c35: Deleting WAL directory at /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/minicluster-data/ts-1/wal/wals/d97e0d08daf642b3b123d9293bffba89
I20260809 02:15:51.452973 18169 ts_tablet_manager.cc:1939] T d97e0d08daf642b3b123d9293bffba89 P c209b3ffbd01442d9a920d60e1049c35: Deleting consensus metadata
I20260809 02:15:51.453414 18171 ts_tablet_manager.cc:1905] T d97e0d08daf642b3b123d9293bffba89 P d9c57cc0f0484098bdf002dd0e84738b: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 09 02:15:51 dist-test-slave-s11t krb5kdc[17266](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241750, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.75.129@KRBTEST.COM
I20260809 02:15:51.456267 18171 ts_tablet_manager.cc:1918] T d97e0d08daf642b3b123d9293bffba89 P d9c57cc0f0484098bdf002dd0e84738b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 02:15:51.456374 18171 log.cc:1196] T d97e0d08daf642b3b123d9293bffba89 P d9c57cc0f0484098bdf002dd0e84738b: Deleting WAL directory at /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241741788589-15662-0/minicluster-data/ts-0/wal/wals/d97e0d08daf642b3b123d9293bffba89
I20260809 02:15:51.456653 18171 ts_tablet_manager.cc:1939] T d97e0d08daf642b3b123d9293bffba89 P d9c57cc0f0484098bdf002dd0e84738b: Deleting consensus metadata
I20260809 02:15:51.457667 17310 catalog_manager.cc:4841] TS d9c57cc0f0484098bdf002dd0e84738b (127.15.75.129:34125): tablet d97e0d08daf642b3b123d9293bffba89 (table test-table [id=7d6e9ae7afb04ac99e196c58139cd82f]) successfully deleted
I20260809 02:15:51.457983 17312 catalog_manager.cc:4841] TS c209b3ffbd01442d9a920d60e1049c35 (127.15.75.130:45793): tablet d97e0d08daf642b3b123d9293bffba89 (table test-table [id=7d6e9ae7afb04ac99e196c58139cd82f]) successfully deleted
I20260809 02:15:51.460141 18153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:51.448816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46772 (local address 127.15.75.129:34125)
0809 02:15:51.448946 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:51.448950 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:51.449442 (+ 492us) server_negotiation.cc:407] Connection header received
0809 02:15:51.449650 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:51.449656 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:51.449709 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:51.449767 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:51.450733 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:51.451529 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:51.452843 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:51.453187 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:51.453297 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:51.456434 (+ 3137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:15:51.456444 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:15:51.456461 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:15:51.456487 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:15:51.458480 (+ 1993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:51.458888 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:51.458900 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:51.458902 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:51.458998 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:51.459252 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:51.459255 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:51.459256 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:51.459551 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:15:51.459753 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:51.459873 (+ 120us) server_negotiation.cc:299] Negotiation successful
0809 02:15:51.460003 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260809 02:15:51.460623 17623 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46772
I20260809 02:15:51.470182 18139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:51.462323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53980 (local address 127.15.75.190:41691)
0809 02:15:51.462457 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:51.462460 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:51.462544 (+ 84us) server_negotiation.cc:407] Connection header received
0809 02:15:51.462636 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:51.462639 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:51.462701 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:51.462781 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:51.463370 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:51.464200 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:51.465057 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:51.465411 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:51.465522 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:51.466758 (+ 1236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:15:51.466762 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:15:51.466766 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:15:51.466795 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:15:51.468334 (+ 1539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:51.468813 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:51.468817 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:51.468819 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:51.468917 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:51.469221 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:51.469226 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:51.469230 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:51.469449 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:15:51.469596 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:51.469870 (+ 274us) server_negotiation.cc:299] Negotiation successful
0809 02:15:51.470016 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
W20260809 02:15:51.470590 17326 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53980
I20260809 02:15:51.471947 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 17431
I20260809 02:15:51.484836 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 17645
I20260809 02:15:51.500444 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 17852
I20260809 02:15:51.508373 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 17288
2026-08-09T02:15:51Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (3157 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:15:51 dist-test-slave-s11t krb5kdc[18181](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:15:51 dist-test-slave-s11t krb5kdc[18181](info): set up 2 sockets
Aug 09 02:15:51 dist-test-slave-s11t krb5kdc[18181](info): commencing operation
krb5kdc: starting...
W20260809 02:15:52.735889 15662 mini_kdc.cc:121] Time spent starting KDC: real 1.201s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Aug 09 02:15:52 dist-test-slave-s11t krb5kdc[18181](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241752, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:15:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:15:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.75.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab.
Entry for principal kudu/127.15.75.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab.
I20260809 02:15:52.855356 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:44355
--webserver_interface=127.15.75.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab
--principal=kudu/127.15.75.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:44705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.75.190:44355
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:15:52.964974 18196 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:15:52.965251 18196 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:15:52.965322 18196 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:15:52.969285 18196 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:15:52.969357 18196 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 02:15:52.969389 18196 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:15:52.969458 18196 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:15:52.969498 18196 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:15:52.969533 18196 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:15:52.974607 18196 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:44705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.75.190:44355
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab
--principal=kudu/127.15.75.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:44355
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.75.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18196
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:15:52.975696 18196 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:15:52.976766 18196 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:15:52.985345 18208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:15:52.985345 18204 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:15:52.985743 18196 server_base.cc:1034] running on GCE node
W20260809 02:15:52.985515 18210 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:15:52.986282 18196 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:15:52.987381 18196 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:15:52.988580 18196 hybrid_clock.cc:648] HybridClock initialized: now 1786241752988551 us; error 40 us; skew 500 ppm
Aug 09 02:15:52 dist-test-slave-s11t krb5kdc[18181](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241752, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:15:52.991498 18196 init.cc:377] Logged in from keytab as kudu/127.15.75.190@KRBTEST.COM (short username kudu)
I20260809 02:15:52.992295 18196 webserver.cc:460] Webserver started at http://127.15.75.190:44569/ using document root <none> and password file <none>
I20260809 02:15:52.992859 18196 fs_manager.cc:359] Metadata directory not provided
I20260809 02:15:52.992926 18196 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:15:52.993091 18196 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:15:52.995034 18196 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/minicluster-data/master-0/data/instance:
uuid: "e222f3caa8a145b9ad29749cb134b63c"
format_stamp: "Formatted at 2026-08-09 02:15:52 on dist-test-slave-s11t"
server_key: "81c63f0c95a19fac1fa70096cd1f515c"
server_key_iv: "d5afad22e780e7476f3338d85bc12f0f"
server_key_version: "encryptionkey@0"
I20260809 02:15:52.995551 18196 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/minicluster-data/master-0/wal/instance:
uuid: "e222f3caa8a145b9ad29749cb134b63c"
format_stamp: "Formatted at 2026-08-09 02:15:52 on dist-test-slave-s11t"
server_key: "81c63f0c95a19fac1fa70096cd1f515c"
server_key_iv: "d5afad22e780e7476f3338d85bc12f0f"
server_key_version: "encryptionkey@0"
I20260809 02:15:52.999205 18196 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260809 02:15:53.001549 18224 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:53.002735 18196 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260809 02:15:53.002863 18196 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/minicluster-data/master-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/minicluster-data/master-0/wal
uuid: "e222f3caa8a145b9ad29749cb134b63c"
format_stamp: "Formatted at 2026-08-09 02:15:52 on dist-test-slave-s11t"
server_key: "81c63f0c95a19fac1fa70096cd1f515c"
server_key_iv: "d5afad22e780e7476f3338d85bc12f0f"
server_key_version: "encryptionkey@0"
I20260809 02:15:53.002969 18196 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:15:53.013442 18196 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:15:53.016526 18196 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:15:53.016762 18196 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:15:53.024539 18196 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.190:44355
I20260809 02:15:53.024575 18319 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.190:44355 every 8 connection(s)
I20260809 02:15:53.025462 18196 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
I20260809 02:15:53.027987 18320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:15:53.031322 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 18196
I20260809 02:15:53.031417 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/minicluster-data/master-0/wal/instance
I20260809 02:15:53.031687 15662 external_mini_cluster.cc:1428] Setting key abec1526bf8bb586358d2abce7357b76
I20260809 02:15:53.033702 18320 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e222f3caa8a145b9ad29749cb134b63c: Bootstrap starting.
I20260809 02:15:53.035866 18320 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e222f3caa8a145b9ad29749cb134b63c: Neither blocks nor log segments found. Creating new log.
I20260809 02:15:53.036592 18320 log.cc:824] T 00000000000000000000000000000000 P e222f3caa8a145b9ad29749cb134b63c: Log is configured to *not* fsync() on all Append() calls
I20260809 02:15:53.038399 18320 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e222f3caa8a145b9ad29749cb134b63c: No bootstrap required, opened a new log
Aug 09 02:15:53 dist-test-slave-s11t krb5kdc[18181](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241752, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:15:53.042466 18320 raft_consensus.cc:348] T 00000000000000000000000000000000 P e222f3caa8a145b9ad29749cb134b63c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e222f3caa8a145b9ad29749cb134b63c" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 44355 } }
I20260809 02:15:53.042662 18320 raft_consensus.cc:374] T 00000000000000000000000000000000 P e222f3caa8a145b9ad29749cb134b63c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:15:53.042737 18320 raft_consensus.cc:729] T 00000000000000000000000000000000 P e222f3caa8a145b9ad29749cb134b63c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e222f3caa8a145b9ad29749cb134b63c, State: Initialized, Role: FOLLOWER
I20260809 02:15:53.043188 18320 consensus_queue.cc:260] T 00000000000000000000000000000000 P e222f3caa8a145b9ad29749cb134b63c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e222f3caa8a145b9ad29749cb134b63c" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 44355 } }
I20260809 02:15:53.043308 18320 raft_consensus.cc:388] T 00000000000000000000000000000000 P e222f3caa8a145b9ad29749cb134b63c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:15:53.043390 18320 raft_consensus.cc:482] T 00000000000000000000000000000000 P e222f3caa8a145b9ad29749cb134b63c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:15:53.043490 18320 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e222f3caa8a145b9ad29749cb134b63c [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:15:53.044294 18320 raft_consensus.cc:504] T 00000000000000000000000000000000 P e222f3caa8a145b9ad29749cb134b63c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e222f3caa8a145b9ad29749cb134b63c" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 44355 } }
I20260809 02:15:53.044703 18320 leader_election.cc:302] T 00000000000000000000000000000000 P e222f3caa8a145b9ad29749cb134b63c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e222f3caa8a145b9ad29749cb134b63c; no voters:
I20260809 02:15:53.045054 18320 leader_election.cc:288] T 00000000000000000000000000000000 P e222f3caa8a145b9ad29749cb134b63c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:15:53.046304 18320 sys_catalog.cc:564] T 00000000000000000000000000000000 P e222f3caa8a145b9ad29749cb134b63c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:15:53.046720 18323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:53.033077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43006 (local address 127.15.75.190:44355)
0809 02:15:53.033491 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:53.033505 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:53.033538 (+ 33us) server_negotiation.cc:407] Connection header received
0809 02:15:53.034237 (+ 699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:53.034257 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:53.034564 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:53.034878 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:53.035663 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:53.036735 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:53.037783 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:53.038291 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:53.038467 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:53.041440 (+ 2973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:15:53.041445 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:15:53.041457 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:15:53.041482 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:15:53.043344 (+ 1862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:53.043868 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:53.043873 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:53.043879 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:53.043999 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:53.044303 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:53.044307 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:53.044309 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:53.044867 (+ 558us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:15:53.045080 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:53.045410 (+ 330us) server_negotiation.cc:299] Negotiation successful
0809 02:15:53.045707 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":127,"threads_started":1}
I20260809 02:15:53.046708 18325 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e222f3caa8a145b9ad29749cb134b63c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:15:53.048728 18325 raft_consensus.cc:686] T 00000000000000000000000000000000 P e222f3caa8a145b9ad29749cb134b63c [term 1 LEADER]: Becoming Leader. State: Replica: e222f3caa8a145b9ad29749cb134b63c, State: Running, Role: LEADER
I20260809 02:15:53.049038 18325 consensus_queue.cc:237] T 00000000000000000000000000000000 P e222f3caa8a145b9ad29749cb134b63c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e222f3caa8a145b9ad29749cb134b63c" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 44355 } }
I20260809 02:15:53.050812 18326 sys_catalog.cc:455] T 00000000000000000000000000000000 P e222f3caa8a145b9ad29749cb134b63c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e222f3caa8a145b9ad29749cb134b63c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e222f3caa8a145b9ad29749cb134b63c" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 44355 } } }
I20260809 02:15:53.050853 18339 sys_catalog.cc:455] T 00000000000000000000000000000000 P e222f3caa8a145b9ad29749cb134b63c [sys.catalog]: SysCatalogTable state changed. Reason: New leader e222f3caa8a145b9ad29749cb134b63c. Latest consensus state: current_term: 1 leader_uuid: "e222f3caa8a145b9ad29749cb134b63c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e222f3caa8a145b9ad29749cb134b63c" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 44355 } } }
I20260809 02:15:53.050972 18326 sys_catalog.cc:458] T 00000000000000000000000000000000 P e222f3caa8a145b9ad29749cb134b63c [sys.catalog]: This master's current role is: LEADER
I20260809 02:15:53.051195 18339 sys_catalog.cc:458] T 00000000000000000000000000000000 P e222f3caa8a145b9ad29749cb134b63c [sys.catalog]: This master's current role is: LEADER
W20260809 02:15:53.054147 18341 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e222f3caa8a145b9ad29749cb134b63c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:15:53.054243 18341 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:15:53.054350 18342 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:15:53.055261 18342 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:15:53.060618 18342 catalog_manager.cc:1347] Generated new cluster ID: b2db222b23a946509f919ed4d9fe0ae9
I20260809 02:15:53.060691 18342 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:15:53.071303 18342 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:15:53.072129 18342 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:15:53.079053 18342 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e222f3caa8a145b9ad29749cb134b63c: Generated new TSK 0
I20260809 02:15:53.079665 18342 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.75.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab.
Entry for principal kudu/127.15.75.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab.
I20260809 02:15:53.111640 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.129:0
--local_ip_for_outbound_sockets=127.15.75.129
--webserver_interface=127.15.75.129
--webserver_port=0
--tserver_master_addrs=127.15.75.190:44355
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab
--principal=kudu/127.15.75.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:44705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:15:53.234414 18349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:15:53.234637 18349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:15:53.234683 18349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:15:53.238574 18349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:15:53.238620 18349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:15:53.238657 18349 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 02:15:53.238672 18349 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 02:15:53.238701 18349 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.129
I20260809 02:15:53.243618 18349 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:44705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab
--principal=kudu/127.15.75.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.75.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.75.190:44355
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18349
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.129
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:15:53.244567 18349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:15:53.245545 18349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:15:53.254925 18357 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:15:53.254931 18356 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:15:53.254948 18360 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:15:53.255192 18349 server_base.cc:1034] running on GCE node
I20260809 02:15:53.255765 18349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:15:53.256358 18349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:15:53.257548 18349 hybrid_clock.cc:648] HybridClock initialized: now 1786241753257516 us; error 39 us; skew 500 ppm
Aug 09 02:15:53 dist-test-slave-s11t krb5kdc[18181](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241753, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:15:53.260388 18349 init.cc:377] Logged in from keytab as kudu/127.15.75.129@KRBTEST.COM (short username kudu)
I20260809 02:15:53.261265 18349 webserver.cc:460] Webserver started at http://127.15.75.129:34025/ using document root <none> and password file <none>
I20260809 02:15:53.261804 18349 fs_manager.cc:359] Metadata directory not provided
I20260809 02:15:53.261873 18349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:15:53.262125 18349 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:15:53.264056 18349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/minicluster-data/ts-0/data/instance:
uuid: "8d22a2768bef4c7d9fc30d0311f732b0"
format_stamp: "Formatted at 2026-08-09 02:15:53 on dist-test-slave-s11t"
server_key: "10481f96f468d3f75505dee28c84b7b9"
server_key_iv: "01c83534ee30c127f255787be7a551d5"
server_key_version: "encryptionkey@0"
I20260809 02:15:53.264611 18349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/minicluster-data/ts-0/wal/instance:
uuid: "8d22a2768bef4c7d9fc30d0311f732b0"
format_stamp: "Formatted at 2026-08-09 02:15:53 on dist-test-slave-s11t"
server_key: "10481f96f468d3f75505dee28c84b7b9"
server_key_iv: "01c83534ee30c127f255787be7a551d5"
server_key_version: "encryptionkey@0"
I20260809 02:15:53.268174 18349 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260809 02:15:53.270510 18373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:53.271697 18349 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 02:15:53.271818 18349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/minicluster-data/ts-0/wal
uuid: "8d22a2768bef4c7d9fc30d0311f732b0"
format_stamp: "Formatted at 2026-08-09 02:15:53 on dist-test-slave-s11t"
server_key: "10481f96f468d3f75505dee28c84b7b9"
server_key_iv: "01c83534ee30c127f255787be7a551d5"
server_key_version: "encryptionkey@0"
I20260809 02:15:53.271924 18349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:15:53.294097 18349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:15:53.297227 18349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:15:53.297438 18349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:15:53.298915 18349 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:15:53.298961 18349 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:53.299000 18349 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:15:53.299058 18349 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 02:15:53 dist-test-slave-s11t krb5kdc[18181](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241753, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.129@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:15:53.310518 18349 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.129:36557
I20260809 02:15:53.310705 18544 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.129:36557 every 8 connection(s)
I20260809 02:15:53.311561 18349 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
I20260809 02:15:53.313853 18323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:53.299912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.129:45847 (local address 127.15.75.190:44355)
0809 02:15:53.300044 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:53.300048 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:53.301247 (+ 1199us) server_negotiation.cc:407] Connection header received
0809 02:15:53.302404 (+ 1157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:53.302407 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:53.302472 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:53.302590 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:53.304259 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:53.305218 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:53.306985 (+ 1767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:53.307386 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:53.307504 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:53.310262 (+ 2758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:15:53.310267 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:15:53.310271 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:15:53.310304 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:15:53.311866 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:53.312460 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:53.312463 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:53.312465 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:53.312546 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:53.312935 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:53.312938 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:53.312939 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:53.313112 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:15:53.313193 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:53.313630 (+ 437us) server_negotiation.cc:299] Negotiation successful
0809 02:15:53.313731 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 02:15:53.314603 18388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:53.300421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:44355 (local address 127.15.75.129:45847)
0809 02:15:53.301087 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:53.301127 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:53.302124 (+ 997us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:53.302826 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:53.302834 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:53.303299 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:53.304059 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:53.304077 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:53.305386 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:53.305390 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:53.306842 (+ 1452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:53.306853 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:53.307548 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:53.307551 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:53.307685 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:53.308308 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:53.308337 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:53.310079 (+ 1742us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:15:53.311984 (+ 1905us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:15:53.311989 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:15:53.312004 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:15:53.312349 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:15:53.312643 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:15:53.312647 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:15:53.312648 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:15:53.312849 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:15:53.313214 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:15:53.313219 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:15:53.313423 (+ 204us) client_negotiation.cc:769] Sending connection context
0809 02:15:53.313567 (+ 144us) client_negotiation.cc:240] Negotiation successful
0809 02:15:53.313796 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":164,"threads_started":1}
I20260809 02:15:53.315907 18546 heartbeater.cc:347] Connected to a master server at 127.15.75.190:44355
I20260809 02:15:53.316066 18546 heartbeater.cc:464] Registering TS with master...
I20260809 02:15:53.316536 18546 heartbeater.cc:511] Master 127.15.75.190:44355 requested a full tablet report, sending...
I20260809 02:15:53.318030 18245 ts_manager.cc:194] Registered new tserver with Master: 8d22a2768bef4c7d9fc30d0311f732b0 (127.15.75.129:36557)
I20260809 02:15:53.318565 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 18349
I20260809 02:15:53.318684 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/minicluster-data/ts-0/wal/instance
I20260809 02:15:53.319000 15662 external_mini_cluster.cc:1428] Setting key 3a6235bcde42f9dd7f2ff4c8a6ae9d93
I20260809 02:15:53.319320 18245 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.75.129@KRBTEST.COM'} at 127.15.75.129:45847
I20260809 02:15:53.326128 18323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:53.318191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.129:46421 (local address 127.15.75.190:44355)
0809 02:15:53.318304 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:53.318308 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:53.318545 (+ 237us) server_negotiation.cc:407] Connection header received
0809 02:15:53.318746 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:53.318750 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:53.318797 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:53.318852 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:53.319451 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:53.320185 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:53.321358 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:53.321631 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:53.321718 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:53.322967 (+ 1249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:15:53.322973 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:15:53.322977 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:15:53.323008 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:15:53.324477 (+ 1469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:53.324971 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:53.324975 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:53.324977 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:53.325055 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:53.325336 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:53.325340 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:53.325342 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:53.325504 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:15:53.325609 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:53.325875 (+ 266us) server_negotiation.cc:299] Negotiation successful
0809 02:15:53.325981 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 02:15:53.326139 18555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:53.318131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:44355 (local address 127.15.75.129:46421)
0809 02:15:53.318461 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:53.318476 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:53.318614 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:53.318993 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:53.318995 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:53.319193 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:53.319311 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:53.319319 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:53.320331 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:53.320334 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:53.321217 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:53.321232 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:53.321762 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:53.321769 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:53.321895 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:53.322359 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:53.322379 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:53.322827 (+ 448us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:15:53.324587 (+ 1760us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:15:53.324591 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:15:53.324593 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:15:53.324862 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:15:53.325149 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:15:53.325152 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:15:53.325154 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:15:53.325243 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:15:53.325604 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:15:53.325607 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:15:53.325702 (+ 95us) client_negotiation.cc:769] Sending connection context
0809 02:15:53.325822 (+ 120us) client_negotiation.cc:240] Negotiation successful
0809 02:15:53.325962 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":94,"threads_started":1}
WARNING: no policy specified for kudu/127.15.75.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab.
Entry for principal kudu/127.15.75.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab.
I20260809 02:15:53.348176 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.130:0
--local_ip_for_outbound_sockets=127.15.75.130
--webserver_interface=127.15.75.130
--webserver_port=0
--tserver_master_addrs=127.15.75.190:44355
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab
--principal=kudu/127.15.75.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:44705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:15:53.456511 18559 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:15:53.456802 18559 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:15:53.456893 18559 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:15:53.460834 18559 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:15:53.460880 18559 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:15:53.460947 18559 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 02:15:53.460979 18559 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 02:15:53.461057 18559 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.130
I20260809 02:15:53.466194 18559 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:44705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab
--principal=kudu/127.15.75.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.75.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.75.190:44355
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18559
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.130
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:15:53.467159 18559 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:15:53.468128 18559 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:15:53.477648 18564 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:15:53.477674 18570 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:15:53.477665 18565 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:15:53.477844 18559 server_base.cc:1034] running on GCE node
I20260809 02:15:53.478314 18559 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:15:53.478909 18559 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:15:53.480084 18559 hybrid_clock.cc:648] HybridClock initialized: now 1786241753480063 us; error 32 us; skew 500 ppm
Aug 09 02:15:53 dist-test-slave-s11t krb5kdc[18181](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241753, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:15:53.482774 18559 init.cc:377] Logged in from keytab as kudu/127.15.75.130@KRBTEST.COM (short username kudu)
I20260809 02:15:53.483664 18559 webserver.cc:460] Webserver started at http://127.15.75.130:40109/ using document root <none> and password file <none>
I20260809 02:15:53.484210 18559 fs_manager.cc:359] Metadata directory not provided
I20260809 02:15:53.484304 18559 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:15:53.484534 18559 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:15:53.486553 18559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/minicluster-data/ts-1/data/instance:
uuid: "09772a64befd467791220629b9d0561e"
format_stamp: "Formatted at 2026-08-09 02:15:53 on dist-test-slave-s11t"
server_key: "155c3c04248bff40ea1d1510039c1b5a"
server_key_iv: "522fd2ed3241d908907b6d3bb5081ca9"
server_key_version: "encryptionkey@0"
I20260809 02:15:53.487125 18559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/minicluster-data/ts-1/wal/instance:
uuid: "09772a64befd467791220629b9d0561e"
format_stamp: "Formatted at 2026-08-09 02:15:53 on dist-test-slave-s11t"
server_key: "155c3c04248bff40ea1d1510039c1b5a"
server_key_iv: "522fd2ed3241d908907b6d3bb5081ca9"
server_key_version: "encryptionkey@0"
I20260809 02:15:53.490723 18559 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 02:15:53.493058 18580 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:53.494168 18559 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 02:15:53.494303 18559 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/minicluster-data/ts-1/wal
uuid: "09772a64befd467791220629b9d0561e"
format_stamp: "Formatted at 2026-08-09 02:15:53 on dist-test-slave-s11t"
server_key: "155c3c04248bff40ea1d1510039c1b5a"
server_key_iv: "522fd2ed3241d908907b6d3bb5081ca9"
server_key_version: "encryptionkey@0"
I20260809 02:15:53.494413 18559 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:15:53.513383 18559 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:15:53.515965 18559 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:15:53.516227 18559 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:15:53.517681 18559 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:15:53.517752 18559 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:53.517817 18559 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:15:53.517867 18559 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 02:15:53 dist-test-slave-s11t krb5kdc[18181](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241753, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.130@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:15:53.529181 18559 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.130:45953
I20260809 02:15:53.529274 18742 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.130:45953 every 8 connection(s)
I20260809 02:15:53.530085 18559 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
I20260809 02:15:53.532058 18323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:53.518583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.130:42603 (local address 127.15.75.190:44355)
0809 02:15:53.518716 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:53.518721 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:53.519756 (+ 1035us) server_negotiation.cc:407] Connection header received
0809 02:15:53.520903 (+ 1147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:53.520909 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:53.520980 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:53.521090 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:53.522443 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:53.522989 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:53.524239 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:53.524475 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:53.524597 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:53.528439 (+ 3842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:15:53.528445 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:15:53.528452 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:15:53.528478 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:15:53.530076 (+ 1598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:53.530624 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:53.530628 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:53.530630 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:53.530739 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:53.531113 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:53.531116 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:53.531117 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:53.531293 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:15:53.531381 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:53.531842 (+ 461us) server_negotiation.cc:299] Negotiation successful
0809 02:15:53.531943 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 02:15:53.532931 18592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:53.519109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:44355 (local address 127.15.75.130:42603)
0809 02:15:53.519577 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:53.519617 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:53.520607 (+ 990us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:53.521298 (+ 691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:53.521309 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:53.521782 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:53.522293 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:53.522309 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:53.523121 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:53.523124 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:53.524105 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:53.524115 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:53.524652 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:53.524656 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:53.524869 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:53.525494 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:53.525517 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:53.528262 (+ 2745us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:15:53.530199 (+ 1937us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:15:53.530206 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:15:53.530220 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:15:53.530477 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:15:53.530851 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:15:53.530855 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:15:53.530856 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:15:53.531011 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:15:53.531396 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:15:53.531401 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:15:53.531604 (+ 203us) client_negotiation.cc:769] Sending connection context
0809 02:15:53.531793 (+ 189us) client_negotiation.cc:240] Negotiation successful
0809 02:15:53.532025 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":121,"threads_started":1}
I20260809 02:15:53.534049 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 18559
I20260809 02:15:53.534116 18743 heartbeater.cc:347] Connected to a master server at 127.15.75.190:44355
I20260809 02:15:53.534173 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/minicluster-data/ts-1/wal/instance
I20260809 02:15:53.534236 18743 heartbeater.cc:464] Registering TS with master...
I20260809 02:15:53.534509 15662 external_mini_cluster.cc:1428] Setting key 3f76162e0ea1d56ac0373f3a29b63170
I20260809 02:15:53.534643 18743 heartbeater.cc:511] Master 127.15.75.190:44355 requested a full tablet report, sending...
I20260809 02:15:53.535650 18249 ts_manager.cc:194] Registered new tserver with Master: 09772a64befd467791220629b9d0561e (127.15.75.130:45953)
I20260809 02:15:53.536346 18249 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.75.130@KRBTEST.COM'} at 127.15.75.130:42603
I20260809 02:15:53.544421 18323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:53.536473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.130:38395 (local address 127.15.75.190:44355)
0809 02:15:53.536674 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:53.536697 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:53.537444 (+ 747us) server_negotiation.cc:407] Connection header received
0809 02:15:53.537605 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:53.537611 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:53.537665 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:53.537749 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:53.538255 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:53.538827 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:53.539785 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:53.539992 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:53.540070 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:53.541396 (+ 1326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:15:53.541399 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:15:53.541401 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:15:53.541422 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:15:53.542796 (+ 1374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:53.543275 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:53.543278 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:53.543279 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:53.543348 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:53.543642 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:53.543644 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:53.543645 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:53.543826 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:15:53.543904 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:53.544224 (+ 320us) server_negotiation.cc:299] Negotiation successful
0809 02:15:53.544311 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
I20260809 02:15:53.544538 18758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:53.537057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:44355 (local address 127.15.75.130:38395)
0809 02:15:53.537328 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:53.537344 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:53.537511 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:53.537788 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:53.537791 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:53.538004 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:53.538115 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:53.538122 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:53.538971 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:53.538974 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:53.539665 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:53.539680 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:53.540110 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:53.540116 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:53.540245 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:53.540705 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:53.540726 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:53.541227 (+ 501us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:15:53.542913 (+ 1686us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:15:53.542918 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:15:53.542922 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:15:53.543181 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:15:53.543444 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:15:53.543447 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:15:53.543449 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:15:53.543549 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:15:53.543929 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:15:53.543935 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:15:53.544041 (+ 106us) client_negotiation.cc:769] Sending connection context
0809 02:15:53.544162 (+ 121us) client_negotiation.cc:240] Negotiation successful
0809 02:15:53.544302 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":88,"threads_started":1}
WARNING: no policy specified for kudu/127.15.75.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab.
Entry for principal kudu/127.15.75.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab.
I20260809 02:15:53.563447 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.131:0
--local_ip_for_outbound_sockets=127.15.75.131
--webserver_interface=127.15.75.131
--webserver_port=0
--tserver_master_addrs=127.15.75.190:44355
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab
--principal=kudu/127.15.75.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:44705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:15:53.673399 18763 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:15:53.673682 18763 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:15:53.673759 18763 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:15:53.677874 18763 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:15:53.677958 18763 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:15:53.678030 18763 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 02:15:53.678085 18763 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 02:15:53.678150 18763 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.131
I20260809 02:15:53.683202 18763 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:44705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab
--principal=kudu/127.15.75.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.75.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.75.190:44355
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18763
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.131
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:15:53.684193 18763 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:15:53.685196 18763 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:15:53.694602 18771 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:15:53.694623 18774 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:15:53.694648 18763 server_base.cc:1034] running on GCE node
W20260809 02:15:53.694602 18770 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:15:53.695209 18763 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:15:53.695842 18763 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:15:53.697029 18763 hybrid_clock.cc:648] HybridClock initialized: now 1786241753697007 us; error 44 us; skew 500 ppm
Aug 09 02:15:53 dist-test-slave-s11t krb5kdc[18181](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241753, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:15:53.699769 18763 init.cc:377] Logged in from keytab as kudu/127.15.75.131@KRBTEST.COM (short username kudu)
I20260809 02:15:53.700574 18763 webserver.cc:460] Webserver started at http://127.15.75.131:36693/ using document root <none> and password file <none>
I20260809 02:15:53.701165 18763 fs_manager.cc:359] Metadata directory not provided
I20260809 02:15:53.701278 18763 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:15:53.701511 18763 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:15:53.703451 18763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/minicluster-data/ts-2/data/instance:
uuid: "d678864a5bbe41fe869538ac35ffeea2"
format_stamp: "Formatted at 2026-08-09 02:15:53 on dist-test-slave-s11t"
server_key: "0f7fe3ef0ba39b961c8734b04a980a60"
server_key_iv: "43b6108215e9ca1b09f618a479e62b28"
server_key_version: "encryptionkey@0"
I20260809 02:15:53.704005 18763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/minicluster-data/ts-2/wal/instance:
uuid: "d678864a5bbe41fe869538ac35ffeea2"
format_stamp: "Formatted at 2026-08-09 02:15:53 on dist-test-slave-s11t"
server_key: "0f7fe3ef0ba39b961c8734b04a980a60"
server_key_iv: "43b6108215e9ca1b09f618a479e62b28"
server_key_version: "encryptionkey@0"
I20260809 02:15:53.708285 18763 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260809 02:15:53.710702 18783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:53.711864 18763 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 02:15:53.712005 18763 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/minicluster-data/ts-2/wal
uuid: "d678864a5bbe41fe869538ac35ffeea2"
format_stamp: "Formatted at 2026-08-09 02:15:53 on dist-test-slave-s11t"
server_key: "0f7fe3ef0ba39b961c8734b04a980a60"
server_key_iv: "43b6108215e9ca1b09f618a479e62b28"
server_key_version: "encryptionkey@0"
I20260809 02:15:53.712116 18763 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:15:53.720638 18763 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:15:53.723238 18763 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:15:53.723429 18763 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:15:53.724819 18763 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:15:53.724886 18763 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:53.724964 18763 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:15:53.725005 18763 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 02:15:53 dist-test-slave-s11t krb5kdc[18181](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241753, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.131@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:15:53.736275 18763 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.131:45385
I20260809 02:15:53.736368 18941 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.131:45385 every 8 connection(s)
I20260809 02:15:53.737159 18763 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
I20260809 02:15:53.739092 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 18763
I20260809 02:15:53.739228 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/minicluster-data/ts-2/wal/instance
I20260809 02:15:53.739557 15662 external_mini_cluster.cc:1428] Setting key 2555c9c52189b1bc36ad1e9a60b2204a
I20260809 02:15:53.740391 18793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:53.725984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:44355 (local address 127.15.75.131:39837)
0809 02:15:53.726463 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:53.726520 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:53.727623 (+ 1103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:53.728460 (+ 837us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:53.728470 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:53.729026 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:53.729689 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:53.729714 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:53.730567 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:53.730571 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:53.732101 (+ 1530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:53.732113 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:53.732936 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:53.732941 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:53.733121 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:53.733683 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:53.733699 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:53.735264 (+ 1565us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:15:53.737352 (+ 2088us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:15:53.737357 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:15:53.737367 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:15:53.737707 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:15:53.738057 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:15:53.738062 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:15:53.738064 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:15:53.738203 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:15:53.738749 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:15:53.738755 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:15:53.739049 (+ 294us) client_negotiation.cc:769] Sending connection context
0809 02:15:53.739275 (+ 226us) client_negotiation.cc:240] Negotiation successful
0809 02:15:53.739593 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":106,"threads_started":1}
I20260809 02:15:53.741102 18323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:53.725657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.131:39837 (local address 127.15.75.190:44355)
0809 02:15:53.725828 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:53.725834 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:53.726641 (+ 807us) server_negotiation.cc:407] Connection header received
0809 02:15:53.727876 (+ 1235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:53.727880 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:53.727948 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:53.728013 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:53.729873 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:53.730427 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:53.732249 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:53.732638 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:53.732879 (+ 241us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:53.735434 (+ 2555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:15:53.735438 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:15:53.735441 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:15:53.735514 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:15:53.737183 (+ 1669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:53.737845 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:53.737849 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:53.737850 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:53.737947 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:53.738425 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:53.738429 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:53.738431 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:53.738626 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:15:53.738754 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:53.740826 (+ 2072us) server_negotiation.cc:299] Negotiation successful
0809 02:15:53.740944 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 02:15:53.742133 18942 heartbeater.cc:347] Connected to a master server at 127.15.75.190:44355
I20260809 02:15:53.742283 18942 heartbeater.cc:464] Registering TS with master...
I20260809 02:15:53.742805 18942 heartbeater.cc:511] Master 127.15.75.190:44355 requested a full tablet report, sending...
I20260809 02:15:53.743764 18255 ts_manager.cc:194] Registered new tserver with Master: d678864a5bbe41fe869538ac35ffeea2 (127.15.75.131:45385)
I20260809 02:15:53.744479 18255 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.75.131@KRBTEST.COM'} at 127.15.75.131:39837
I20260809 02:15:53.750991 18948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:53.744168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:44355 (local address 127.15.75.131:56661)
0809 02:15:53.744559 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:53.744573 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:53.744726 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:53.744979 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:53.744982 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:53.745169 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:53.745262 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:53.745267 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:53.746221 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:53.746223 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:53.746904 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:53.746914 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:53.747367 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:53.747370 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:53.747444 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:53.747876 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:53.747887 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:53.748164 (+ 277us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:15:53.749855 (+ 1691us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:15:53.749857 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:15:53.749859 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:15:53.750037 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:15:53.750265 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:15:53.750267 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:15:53.750268 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:15:53.750338 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:15:53.750634 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:15:53.750636 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:15:53.750703 (+ 67us) client_negotiation.cc:769] Sending connection context
0809 02:15:53.750779 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 02:15:53.750873 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":102,"threads_started":1}
I20260809 02:15:53.751010 18323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:53.744278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.131:56661 (local address 127.15.75.190:44355)
0809 02:15:53.744402 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:53.744407 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:53.744636 (+ 229us) server_negotiation.cc:407] Connection header received
0809 02:15:53.744818 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:53.744821 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:53.744869 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:53.744947 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:53.745380 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:53.746101 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:53.747035 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:53.747253 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:53.747369 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:53.748283 (+ 914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:15:53.748286 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:15:53.748287 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:15:53.748308 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:15:53.749767 (+ 1459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:53.750117 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:53.750120 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:53.750121 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:53.750189 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:53.750412 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:53.750415 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:53.750416 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:53.750560 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:15:53.750632 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:53.750812 (+ 180us) server_negotiation.cc:299] Negotiation successful
0809 02:15:53.750911 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 02:15:53.753540 15662 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 02:15:53 dist-test-slave-s11t krb5kdc[18181](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241753, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 02:15:53 dist-test-slave-s11t krb5kdc[18181](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241753, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:15:53.779743 18323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:53.771833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43022 (local address 127.15.75.190:44355)
0809 02:15:53.771964 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:53.771967 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:53.771979 (+ 12us) server_negotiation.cc:407] Connection header received
0809 02:15:53.772012 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:53.772014 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:53.772047 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:53.772112 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:53.772807 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:53.773300 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:53.774117 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:53.774312 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:53.774384 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:53.776822 (+ 2438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:15:53.776825 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:15:53.776826 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:15:53.776848 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:15:53.778293 (+ 1445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:53.778683 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:53.778685 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:53.778687 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:53.778758 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:53.779001 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:53.779003 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:53.779004 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:53.779155 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:15:53.779249 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:53.779449 (+ 200us) server_negotiation.cc:299] Negotiation successful
0809 02:15:53.779537 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 02:15:53.782591 18255 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43022:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:15:53.785542 18255 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:15:53.799619 18966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:53.793644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.129:36557 (local address 127.0.0.1:53630)
0809 02:15:53.794394 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:53.794443 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:53.794567 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:53.795643 (+ 1076us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:53.795648 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:53.795671 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:15:53.795816 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:53.795827 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:53.797082 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:53.797085 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:53.798285 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:53.798295 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:53.799236 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:53.799238 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:53.799318 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:53.799347 (+ 29us) client_negotiation.cc:769] Sending connection context
0809 02:15:53.799431 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 02:15:53.799512 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":575,"thread_start_us":74,"threads_started":1}
I20260809 02:15:53.799690 18971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:53.793795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53630 (local address 127.15.75.129:36557)
0809 02:15:53.794878 (+ 1083us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:53.795038 (+ 160us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:53.795073 (+ 35us) server_negotiation.cc:407] Connection header received
0809 02:15:53.795221 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:53.795229 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:53.795479 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:53.795639 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:15:53.795958 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:53.796956 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:53.798424 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:53.799123 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:53.799231 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:53.799397 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:53.799457 (+ 60us) server_negotiation.cc:299] Negotiation successful
0809 02:15:53.799545 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":915,"thread_start_us":65,"threads_started":1}
I20260809 02:15:53.801355 18976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:53.794077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.131:45385 (local address 127.0.0.1:55542)
0809 02:15:53.795274 (+ 1197us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:53.795290 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:53.795388 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:53.796064 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:53.796066 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:53.796084 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:15:53.796157 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:53.796163 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:53.797546 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:53.797549 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:53.798928 (+ 1379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:53.798940 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:53.800929 (+ 1989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:53.800935 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:53.801044 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:53.801066 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 02:15:53.801138 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 02:15:53.801221 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":817,"mutex_wait_us":60,"spinlock_wait_cycles":678144,"thread_start_us":111,"threads_started":1}
I20260809 02:15:53.801507 18978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:53.794404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48474 (local address 127.15.75.130:45953)
0809 02:15:53.795178 (+ 774us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:53.795193 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:53.795267 (+ 74us) server_negotiation.cc:407] Connection header received
0809 02:15:53.795509 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:53.795515 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:53.795697 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:53.795842 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:15:53.796085 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:53.797291 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:53.799882 (+ 2591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:53.800867 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:53.800980 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:53.801135 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:53.801255 (+ 120us) server_negotiation.cc:299] Negotiation successful
0809 02:15:53.801349 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":100,"threads_started":1}
I20260809 02:15:53.801750 18974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:53.793816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.130:45953 (local address 127.0.0.1:48474)
0809 02:15:53.795158 (+ 1342us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:53.795175 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:53.795314 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:53.795822 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:53.795825 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:53.795843 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:15:53.795970 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:53.795976 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:53.797435 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:53.797439 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:53.798928 (+ 1489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:53.798940 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:53.800998 (+ 2058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:53.801002 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:53.801122 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:53.801146 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 02:15:53.801226 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 02:15:53.801297 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1062,"mutex_wait_us":210,"thread_start_us":112,"threads_started":1}
I20260809 02:15:53.801868 18979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:53.794493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55542 (local address 127.15.75.131:45385)
0809 02:15:53.795167 (+ 674us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:53.795184 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:53.795359 (+ 175us) server_negotiation.cc:407] Connection header received
0809 02:15:53.795515 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:53.795525 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:53.795740 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:53.795921 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:15:53.796290 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:53.797424 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:53.799047 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:53.800551 (+ 1504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:53.800664 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:53.800856 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:53.801605 (+ 749us) server_negotiation.cc:299] Negotiation successful
0809 02:15:53.801709 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":585,"thread_start_us":105,"threads_started":1}
I20260809 02:15:53.802929 18447 tablet_service.cc:1467] Processing CreateTablet for tablet 03c68b4fb78946d29fb07ebf4b9e1041 (DEFAULT_TABLE table=test-table [id=f4c79ab4e364470893686ae03f6c07a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:15:53.804119 18447 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03c68b4fb78946d29fb07ebf4b9e1041. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:15:53.804195 18648 tablet_service.cc:1467] Processing CreateTablet for tablet 03c68b4fb78946d29fb07ebf4b9e1041 (DEFAULT_TABLE table=test-table [id=f4c79ab4e364470893686ae03f6c07a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:15:53.804471 18857 tablet_service.cc:1467] Processing CreateTablet for tablet 03c68b4fb78946d29fb07ebf4b9e1041 (DEFAULT_TABLE table=test-table [id=f4c79ab4e364470893686ae03f6c07a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:15:53.805152 18648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03c68b4fb78946d29fb07ebf4b9e1041. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:15:53.805464 18857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03c68b4fb78946d29fb07ebf4b9e1041. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:15:53.811370 18981 tablet_bootstrap.cc:492] T 03c68b4fb78946d29fb07ebf4b9e1041 P 09772a64befd467791220629b9d0561e: Bootstrap starting.
I20260809 02:15:53.812510 18986 tablet_bootstrap.cc:492] T 03c68b4fb78946d29fb07ebf4b9e1041 P 8d22a2768bef4c7d9fc30d0311f732b0: Bootstrap starting.
I20260809 02:15:53.813264 18987 tablet_bootstrap.cc:492] T 03c68b4fb78946d29fb07ebf4b9e1041 P d678864a5bbe41fe869538ac35ffeea2: Bootstrap starting.
I20260809 02:15:53.814054 18981 tablet_bootstrap.cc:654] T 03c68b4fb78946d29fb07ebf4b9e1041 P 09772a64befd467791220629b9d0561e: Neither blocks nor log segments found. Creating new log.
I20260809 02:15:53.814692 18986 tablet_bootstrap.cc:654] T 03c68b4fb78946d29fb07ebf4b9e1041 P 8d22a2768bef4c7d9fc30d0311f732b0: Neither blocks nor log segments found. Creating new log.
I20260809 02:15:53.814996 18981 log.cc:824] T 03c68b4fb78946d29fb07ebf4b9e1041 P 09772a64befd467791220629b9d0561e: Log is configured to *not* fsync() on all Append() calls
I20260809 02:15:53.815320 18987 tablet_bootstrap.cc:654] T 03c68b4fb78946d29fb07ebf4b9e1041 P d678864a5bbe41fe869538ac35ffeea2: Neither blocks nor log segments found. Creating new log.
I20260809 02:15:53.815570 18986 log.cc:824] T 03c68b4fb78946d29fb07ebf4b9e1041 P 8d22a2768bef4c7d9fc30d0311f732b0: Log is configured to *not* fsync() on all Append() calls
I20260809 02:15:53.816219 18987 log.cc:824] T 03c68b4fb78946d29fb07ebf4b9e1041 P d678864a5bbe41fe869538ac35ffeea2: Log is configured to *not* fsync() on all Append() calls
I20260809 02:15:53.817418 18981 tablet_bootstrap.cc:492] T 03c68b4fb78946d29fb07ebf4b9e1041 P 09772a64befd467791220629b9d0561e: No bootstrap required, opened a new log
I20260809 02:15:53.817437 18986 tablet_bootstrap.cc:492] T 03c68b4fb78946d29fb07ebf4b9e1041 P 8d22a2768bef4c7d9fc30d0311f732b0: No bootstrap required, opened a new log
I20260809 02:15:53.817617 18981 ts_tablet_manager.cc:1397] T 03c68b4fb78946d29fb07ebf4b9e1041 P 09772a64befd467791220629b9d0561e: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260809 02:15:53.817616 18986 ts_tablet_manager.cc:1397] T 03c68b4fb78946d29fb07ebf4b9e1041 P 8d22a2768bef4c7d9fc30d0311f732b0: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 02:15:53.817988 18987 tablet_bootstrap.cc:492] T 03c68b4fb78946d29fb07ebf4b9e1041 P d678864a5bbe41fe869538ac35ffeea2: No bootstrap required, opened a new log
I20260809 02:15:53.818185 18987 ts_tablet_manager.cc:1397] T 03c68b4fb78946d29fb07ebf4b9e1041 P d678864a5bbe41fe869538ac35ffeea2: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260809 02:15:53.820933 18987 raft_consensus.cc:348] T 03c68b4fb78946d29fb07ebf4b9e1041 P d678864a5bbe41fe869538ac35ffeea2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d22a2768bef4c7d9fc30d0311f732b0" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 36557 } } peers { permanent_uuid: "09772a64befd467791220629b9d0561e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45953 } } peers { permanent_uuid: "d678864a5bbe41fe869538ac35ffeea2" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45385 } }
I20260809 02:15:53.821161 18987 raft_consensus.cc:374] T 03c68b4fb78946d29fb07ebf4b9e1041 P d678864a5bbe41fe869538ac35ffeea2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:15:53.821215 18987 raft_consensus.cc:729] T 03c68b4fb78946d29fb07ebf4b9e1041 P d678864a5bbe41fe869538ac35ffeea2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d678864a5bbe41fe869538ac35ffeea2, State: Initialized, Role: FOLLOWER
I20260809 02:15:53.821436 18986 raft_consensus.cc:348] T 03c68b4fb78946d29fb07ebf4b9e1041 P 8d22a2768bef4c7d9fc30d0311f732b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d22a2768bef4c7d9fc30d0311f732b0" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 36557 } } peers { permanent_uuid: "09772a64befd467791220629b9d0561e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45953 } } peers { permanent_uuid: "d678864a5bbe41fe869538ac35ffeea2" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45385 } }
I20260809 02:15:53.821436 18981 raft_consensus.cc:348] T 03c68b4fb78946d29fb07ebf4b9e1041 P 09772a64befd467791220629b9d0561e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d22a2768bef4c7d9fc30d0311f732b0" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 36557 } } peers { permanent_uuid: "09772a64befd467791220629b9d0561e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45953 } } peers { permanent_uuid: "d678864a5bbe41fe869538ac35ffeea2" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45385 } }
I20260809 02:15:53.821671 18986 raft_consensus.cc:374] T 03c68b4fb78946d29fb07ebf4b9e1041 P 8d22a2768bef4c7d9fc30d0311f732b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:15:53.821671 18981 raft_consensus.cc:374] T 03c68b4fb78946d29fb07ebf4b9e1041 P 09772a64befd467791220629b9d0561e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:15:53.821720 18986 raft_consensus.cc:729] T 03c68b4fb78946d29fb07ebf4b9e1041 P 8d22a2768bef4c7d9fc30d0311f732b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d22a2768bef4c7d9fc30d0311f732b0, State: Initialized, Role: FOLLOWER
I20260809 02:15:53.821720 18981 raft_consensus.cc:729] T 03c68b4fb78946d29fb07ebf4b9e1041 P 09772a64befd467791220629b9d0561e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09772a64befd467791220629b9d0561e, State: Initialized, Role: FOLLOWER
I20260809 02:15:53.821698 18987 consensus_queue.cc:260] T 03c68b4fb78946d29fb07ebf4b9e1041 P d678864a5bbe41fe869538ac35ffeea2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d22a2768bef4c7d9fc30d0311f732b0" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 36557 } } peers { permanent_uuid: "09772a64befd467791220629b9d0561e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45953 } } peers { permanent_uuid: "d678864a5bbe41fe869538ac35ffeea2" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45385 } }
I20260809 02:15:53.822125 18981 consensus_queue.cc:260] T 03c68b4fb78946d29fb07ebf4b9e1041 P 09772a64befd467791220629b9d0561e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d22a2768bef4c7d9fc30d0311f732b0" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 36557 } } peers { permanent_uuid: "09772a64befd467791220629b9d0561e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45953 } } peers { permanent_uuid: "d678864a5bbe41fe869538ac35ffeea2" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45385 } }
I20260809 02:15:53.822125 18986 consensus_queue.cc:260] T 03c68b4fb78946d29fb07ebf4b9e1041 P 8d22a2768bef4c7d9fc30d0311f732b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d22a2768bef4c7d9fc30d0311f732b0" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 36557 } } peers { permanent_uuid: "09772a64befd467791220629b9d0561e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45953 } } peers { permanent_uuid: "d678864a5bbe41fe869538ac35ffeea2" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45385 } }
I20260809 02:15:53.822785 18987 ts_tablet_manager.cc:1428] T 03c68b4fb78946d29fb07ebf4b9e1041 P d678864a5bbe41fe869538ac35ffeea2: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260809 02:15:53.822799 18546 heartbeater.cc:503] Master 127.15.75.190:44355 was elected leader, sending a full tablet report...
I20260809 02:15:53.823081 18986 ts_tablet_manager.cc:1428] T 03c68b4fb78946d29fb07ebf4b9e1041 P 8d22a2768bef4c7d9fc30d0311f732b0: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260809 02:15:53.823132 18942 heartbeater.cc:503] Master 127.15.75.190:44355 was elected leader, sending a full tablet report...
I20260809 02:15:53.823668 18981 ts_tablet_manager.cc:1428] T 03c68b4fb78946d29fb07ebf4b9e1041 P 09772a64befd467791220629b9d0561e: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 02:15:53.823977 18743 heartbeater.cc:503] Master 127.15.75.190:44355 was elected leader, sending a full tablet report...
I20260809 02:15:53.884956 18995 raft_consensus.cc:482] T 03c68b4fb78946d29fb07ebf4b9e1041 P 8d22a2768bef4c7d9fc30d0311f732b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:15:53.885174 18995 raft_consensus.cc:504] T 03c68b4fb78946d29fb07ebf4b9e1041 P 8d22a2768bef4c7d9fc30d0311f732b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d22a2768bef4c7d9fc30d0311f732b0" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 36557 } } peers { permanent_uuid: "09772a64befd467791220629b9d0561e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45953 } } peers { permanent_uuid: "d678864a5bbe41fe869538ac35ffeea2" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45385 } }
I20260809 02:15:53.886303 18995 leader_election.cc:288] T 03c68b4fb78946d29fb07ebf4b9e1041 P 8d22a2768bef4c7d9fc30d0311f732b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 09772a64befd467791220629b9d0561e (127.15.75.130:45953), d678864a5bbe41fe869538ac35ffeea2 (127.15.75.131:45385)
I20260809 02:15:53.890413 18979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:53.886692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.129:54411 (local address 127.15.75.131:45385)
0809 02:15:53.886909 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:53.886914 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:53.886983 (+ 69us) server_negotiation.cc:407] Connection header received
0809 02:15:53.887213 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:53.887216 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:53.887265 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:53.887343 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:15:53.887743 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:53.888279 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:53.889417 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:53.889826 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:53.889918 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:53.889999 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:53.890197 (+ 198us) server_negotiation.cc:299] Negotiation successful
0809 02:15:53.890275 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129}
I20260809 02:15:53.890424 19004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:53.886548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.131:45385 (local address 127.15.75.129:54411)
0809 02:15:53.886897 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:53.886912 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:53.887078 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:53.887379 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:53.887382 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:53.887402 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:15:53.887554 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:53.887561 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:53.888401 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:53.888408 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:53.889286 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:53.889295 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:53.889963 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:53.889967 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:53.890056 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:53.890087 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 02:15:53.890157 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 02:15:53.890246 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"mutex_wait_us":32,"spinlock_wait_cycles":3328,"thread_start_us":136,"threads_started":1}
I20260809 02:15:53.890453 18978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:53.886870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.129:42677 (local address 127.15.75.130:45953)
0809 02:15:53.887000 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:53.887006 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:53.887022 (+ 16us) server_negotiation.cc:407] Connection header received
0809 02:15:53.887244 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:53.887247 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:53.887298 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:53.887348 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:15:53.887802 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:53.888287 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:53.889460 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:53.889861 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:53.890001 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:53.890053 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:53.890255 (+ 202us) server_negotiation.cc:299] Negotiation successful
0809 02:15:53.890333 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 02:15:53.890431 19001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:53.886510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.130:45953 (local address 127.15.75.129:42677)
0809 02:15:53.886951 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:53.886966 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:53.887093 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:53.887522 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:53.887528 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:53.887545 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:15:53.887665 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:53.887671 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:53.888408 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:53.888411 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:53.889303 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:53.889312 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:53.890007 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:53.890012 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:53.890107 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:53.890124 (+ 17us) client_negotiation.cc:769] Sending connection context
0809 02:15:53.890221 (+ 97us) client_negotiation.cc:240] Negotiation successful
0809 02:15:53.890280 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"mutex_wait_us":58,"spinlock_wait_cycles":14976,"thread_start_us":120,"threads_started":1}
I20260809 02:15:53.891312 18886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03c68b4fb78946d29fb07ebf4b9e1041" candidate_uuid: "8d22a2768bef4c7d9fc30d0311f732b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d678864a5bbe41fe869538ac35ffeea2" is_pre_election: true
I20260809 02:15:53.891641 18886 raft_consensus.cc:2454] T 03c68b4fb78946d29fb07ebf4b9e1041 P d678864a5bbe41fe869538ac35ffeea2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d22a2768bef4c7d9fc30d0311f732b0 in term 0.
I20260809 02:15:53.891642 18678 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03c68b4fb78946d29fb07ebf4b9e1041" candidate_uuid: "8d22a2768bef4c7d9fc30d0311f732b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09772a64befd467791220629b9d0561e" is_pre_election: true
I20260809 02:15:53.891911 18678 raft_consensus.cc:2454] T 03c68b4fb78946d29fb07ebf4b9e1041 P 09772a64befd467791220629b9d0561e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d22a2768bef4c7d9fc30d0311f732b0 in term 0.
I20260809 02:15:53.892115 18374 leader_election.cc:302] T 03c68b4fb78946d29fb07ebf4b9e1041 P 8d22a2768bef4c7d9fc30d0311f732b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d22a2768bef4c7d9fc30d0311f732b0, d678864a5bbe41fe869538ac35ffeea2; no voters:
I20260809 02:15:53.892410 18995 raft_consensus.cc:2781] T 03c68b4fb78946d29fb07ebf4b9e1041 P 8d22a2768bef4c7d9fc30d0311f732b0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:15:53.892509 18995 raft_consensus.cc:482] T 03c68b4fb78946d29fb07ebf4b9e1041 P 8d22a2768bef4c7d9fc30d0311f732b0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:15:53.892572 18995 raft_consensus.cc:3037] T 03c68b4fb78946d29fb07ebf4b9e1041 P 8d22a2768bef4c7d9fc30d0311f732b0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:15:53.893631 18995 raft_consensus.cc:504] T 03c68b4fb78946d29fb07ebf4b9e1041 P 8d22a2768bef4c7d9fc30d0311f732b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d22a2768bef4c7d9fc30d0311f732b0" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 36557 } } peers { permanent_uuid: "09772a64befd467791220629b9d0561e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45953 } } peers { permanent_uuid: "d678864a5bbe41fe869538ac35ffeea2" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45385 } }
I20260809 02:15:53.893975 18995 leader_election.cc:288] T 03c68b4fb78946d29fb07ebf4b9e1041 P 8d22a2768bef4c7d9fc30d0311f732b0 [CANDIDATE]: Term 1 election: Requested vote from peers 09772a64befd467791220629b9d0561e (127.15.75.130:45953), d678864a5bbe41fe869538ac35ffeea2 (127.15.75.131:45385)
I20260809 02:15:53.894335 18886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03c68b4fb78946d29fb07ebf4b9e1041" candidate_uuid: "8d22a2768bef4c7d9fc30d0311f732b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d678864a5bbe41fe869538ac35ffeea2"
I20260809 02:15:53.894467 18678 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03c68b4fb78946d29fb07ebf4b9e1041" candidate_uuid: "8d22a2768bef4c7d9fc30d0311f732b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09772a64befd467791220629b9d0561e"
I20260809 02:15:53.894480 18886 raft_consensus.cc:3037] T 03c68b4fb78946d29fb07ebf4b9e1041 P d678864a5bbe41fe869538ac35ffeea2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:15:53.894588 18678 raft_consensus.cc:3037] T 03c68b4fb78946d29fb07ebf4b9e1041 P 09772a64befd467791220629b9d0561e [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:15:53.895944 18678 raft_consensus.cc:2454] T 03c68b4fb78946d29fb07ebf4b9e1041 P 09772a64befd467791220629b9d0561e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d22a2768bef4c7d9fc30d0311f732b0 in term 1.
I20260809 02:15:53.896251 18886 raft_consensus.cc:2454] T 03c68b4fb78946d29fb07ebf4b9e1041 P d678864a5bbe41fe869538ac35ffeea2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d22a2768bef4c7d9fc30d0311f732b0 in term 1.
I20260809 02:15:53.896255 18377 leader_election.cc:302] T 03c68b4fb78946d29fb07ebf4b9e1041 P 8d22a2768bef4c7d9fc30d0311f732b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09772a64befd467791220629b9d0561e, 8d22a2768bef4c7d9fc30d0311f732b0; no voters:
I20260809 02:15:53.896437 18995 raft_consensus.cc:2781] T 03c68b4fb78946d29fb07ebf4b9e1041 P 8d22a2768bef4c7d9fc30d0311f732b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:15:53.896695 18995 raft_consensus.cc:686] T 03c68b4fb78946d29fb07ebf4b9e1041 P 8d22a2768bef4c7d9fc30d0311f732b0 [term 1 LEADER]: Becoming Leader. State: Replica: 8d22a2768bef4c7d9fc30d0311f732b0, State: Running, Role: LEADER
I20260809 02:15:53.897042 18995 consensus_queue.cc:237] T 03c68b4fb78946d29fb07ebf4b9e1041 P 8d22a2768bef4c7d9fc30d0311f732b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d22a2768bef4c7d9fc30d0311f732b0" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 36557 } } peers { permanent_uuid: "09772a64befd467791220629b9d0561e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45953 } } peers { permanent_uuid: "d678864a5bbe41fe869538ac35ffeea2" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45385 } }
I20260809 02:15:53.900146 18255 catalog_manager.cc:5495] T 03c68b4fb78946d29fb07ebf4b9e1041 P 8d22a2768bef4c7d9fc30d0311f732b0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8d22a2768bef4c7d9fc30d0311f732b0 (127.15.75.129). New cstate: current_term: 1 leader_uuid: "8d22a2768bef4c7d9fc30d0311f732b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d22a2768bef4c7d9fc30d0311f732b0" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 36557 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09772a64befd467791220629b9d0561e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d678864a5bbe41fe869538ac35ffeea2" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45385 } health_report { overall_health: UNKNOWN } } }
Aug 09 02:15:53 dist-test-slave-s11t krb5kdc[18181](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241752, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.190@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:15:53.954771 19022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:53.946473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:44355 (local address 127.0.0.1:43034)
0809 02:15:53.946730 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:53.946743 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:53.946876 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:53.947118 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:53.947121 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:53.947615 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:53.947725 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:53.947732 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:53.948578 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:53.948580 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:53.949194 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:53.949203 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:53.949611 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:53.949614 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:53.949677 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:53.950254 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:53.950266 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:53.951670 (+ 1404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:15:53.953422 (+ 1752us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:15:53.953430 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:15:53.953435 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:15:53.953646 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:15:53.953876 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:15:53.953878 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:15:53.953880 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:15:53.953953 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:15:53.954269 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:15:53.954274 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:15:53.954381 (+ 107us) client_negotiation.cc:769] Sending connection context
0809 02:15:53.954471 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 02:15:53.954590 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":84,"threads_started":1}
I20260809 02:15:53.954774 18323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:53.946494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43034 (local address 127.15.75.190:44355)
0809 02:15:53.946633 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:53.946639 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:53.946815 (+ 176us) server_negotiation.cc:407] Connection header received
0809 02:15:53.946968 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:53.946971 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:53.947017 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:53.947100 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:53.947871 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:53.948480 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:53.949302 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:53.949516 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:53.949591 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:53.951794 (+ 2203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:15:53.951797 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:15:53.951799 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:15:53.951820 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:15:53.953321 (+ 1501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:53.953726 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:53.953729 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:53.953730 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:53.953800 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:53.954026 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:53.954029 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:53.954030 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:53.954184 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:15:53.954257 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:53.954530 (+ 273us) server_negotiation.cc:299] Negotiation successful
0809 02:15:53.954645 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":18,"server-negotiator.queue_time_us":45}
I20260809 02:15:53.958467 18255 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.15.75.190@KRBTEST.COM'} at 127.0.0.1:43034:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 02:15:53.959165 18255 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:15:53.964287 18857 tablet_service.cc:1467] Processing CreateTablet for tablet 851019929b3b4bc69f3a7d616a4c2df1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2c5a9708f87e4e98accf4d1e2649969e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 02:15:53.964147 18447 tablet_service.cc:1467] Processing CreateTablet for tablet 851019929b3b4bc69f3a7d616a4c2df1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2c5a9708f87e4e98accf4d1e2649969e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 02:15:53.964365 18648 tablet_service.cc:1467] Processing CreateTablet for tablet 851019929b3b4bc69f3a7d616a4c2df1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2c5a9708f87e4e98accf4d1e2649969e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 02:15:53.964932 18857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 851019929b3b4bc69f3a7d616a4c2df1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:15:53.964933 18648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 851019929b3b4bc69f3a7d616a4c2df1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:15:53.964943 18447 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 851019929b3b4bc69f3a7d616a4c2df1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:15:53.967283 18987 tablet_bootstrap.cc:492] T 851019929b3b4bc69f3a7d616a4c2df1 P d678864a5bbe41fe869538ac35ffeea2: Bootstrap starting.
I20260809 02:15:53.968107 18981 tablet_bootstrap.cc:492] T 851019929b3b4bc69f3a7d616a4c2df1 P 09772a64befd467791220629b9d0561e: Bootstrap starting.
I20260809 02:15:53.968199 18986 tablet_bootstrap.cc:492] T 851019929b3b4bc69f3a7d616a4c2df1 P 8d22a2768bef4c7d9fc30d0311f732b0: Bootstrap starting.
I20260809 02:15:53.968359 18987 tablet_bootstrap.cc:654] T 851019929b3b4bc69f3a7d616a4c2df1 P d678864a5bbe41fe869538ac35ffeea2: Neither blocks nor log segments found. Creating new log.
I20260809 02:15:53.969105 18981 tablet_bootstrap.cc:654] T 851019929b3b4bc69f3a7d616a4c2df1 P 09772a64befd467791220629b9d0561e: Neither blocks nor log segments found. Creating new log.
I20260809 02:15:53.969321 18986 tablet_bootstrap.cc:654] T 851019929b3b4bc69f3a7d616a4c2df1 P 8d22a2768bef4c7d9fc30d0311f732b0: Neither blocks nor log segments found. Creating new log.
I20260809 02:15:53.969820 18987 tablet_bootstrap.cc:492] T 851019929b3b4bc69f3a7d616a4c2df1 P d678864a5bbe41fe869538ac35ffeea2: No bootstrap required, opened a new log
I20260809 02:15:53.969915 18987 ts_tablet_manager.cc:1397] T 851019929b3b4bc69f3a7d616a4c2df1 P d678864a5bbe41fe869538ac35ffeea2: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260809 02:15:53.970463 18981 tablet_bootstrap.cc:492] T 851019929b3b4bc69f3a7d616a4c2df1 P 09772a64befd467791220629b9d0561e: No bootstrap required, opened a new log
I20260809 02:15:53.970427 18987 raft_consensus.cc:348] T 851019929b3b4bc69f3a7d616a4c2df1 P d678864a5bbe41fe869538ac35ffeea2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d22a2768bef4c7d9fc30d0311f732b0" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 36557 } } peers { permanent_uuid: "09772a64befd467791220629b9d0561e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45953 } } peers { permanent_uuid: "d678864a5bbe41fe869538ac35ffeea2" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45385 } }
I20260809 02:15:53.970551 18987 raft_consensus.cc:374] T 851019929b3b4bc69f3a7d616a4c2df1 P d678864a5bbe41fe869538ac35ffeea2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:15:53.970573 18981 ts_tablet_manager.cc:1397] T 851019929b3b4bc69f3a7d616a4c2df1 P 09772a64befd467791220629b9d0561e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260809 02:15:53.970595 18987 raft_consensus.cc:729] T 851019929b3b4bc69f3a7d616a4c2df1 P d678864a5bbe41fe869538ac35ffeea2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d678864a5bbe41fe869538ac35ffeea2, State: Initialized, Role: FOLLOWER
I20260809 02:15:53.970737 18987 consensus_queue.cc:260] T 851019929b3b4bc69f3a7d616a4c2df1 P d678864a5bbe41fe869538ac35ffeea2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d22a2768bef4c7d9fc30d0311f732b0" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 36557 } } peers { permanent_uuid: "09772a64befd467791220629b9d0561e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45953 } } peers { permanent_uuid: "d678864a5bbe41fe869538ac35ffeea2" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45385 } }
I20260809 02:15:53.971019 18981 raft_consensus.cc:348] T 851019929b3b4bc69f3a7d616a4c2df1 P 09772a64befd467791220629b9d0561e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d22a2768bef4c7d9fc30d0311f732b0" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 36557 } } peers { permanent_uuid: "09772a64befd467791220629b9d0561e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45953 } } peers { permanent_uuid: "d678864a5bbe41fe869538ac35ffeea2" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45385 } }
I20260809 02:15:53.971119 18981 raft_consensus.cc:374] T 851019929b3b4bc69f3a7d616a4c2df1 P 09772a64befd467791220629b9d0561e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:15:53.971061 18994 tablet_replica.cc:439] T 851019929b3b4bc69f3a7d616a4c2df1 P d678864a5bbe41fe869538ac35ffeea2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d22a2768bef4c7d9fc30d0311f732b0" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 36557 } } peers { permanent_uuid: "09772a64befd467791220629b9d0561e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45953 } } peers { permanent_uuid: "d678864a5bbe41fe869538ac35ffeea2" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45385 } } }
I20260809 02:15:53.971155 18981 raft_consensus.cc:729] T 851019929b3b4bc69f3a7d616a4c2df1 P 09772a64befd467791220629b9d0561e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09772a64befd467791220629b9d0561e, State: Initialized, Role: FOLLOWER
I20260809 02:15:53.971189 18994 tablet_replica.cc:442] T 851019929b3b4bc69f3a7d616a4c2df1 P d678864a5bbe41fe869538ac35ffeea2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 02:15:53.971280 18986 tablet_bootstrap.cc:492] T 851019929b3b4bc69f3a7d616a4c2df1 P 8d22a2768bef4c7d9fc30d0311f732b0: No bootstrap required, opened a new log
I20260809 02:15:53.971271 18981 consensus_queue.cc:260] T 851019929b3b4bc69f3a7d616a4c2df1 P 09772a64befd467791220629b9d0561e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d22a2768bef4c7d9fc30d0311f732b0" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 36557 } } peers { permanent_uuid: "09772a64befd467791220629b9d0561e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45953 } } peers { permanent_uuid: "d678864a5bbe41fe869538ac35ffeea2" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45385 } }
I20260809 02:15:53.971352 18986 ts_tablet_manager.cc:1397] T 851019929b3b4bc69f3a7d616a4c2df1 P 8d22a2768bef4c7d9fc30d0311f732b0: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 02:15:53.971477 18987 ts_tablet_manager.cc:1428] T 851019929b3b4bc69f3a7d616a4c2df1 P d678864a5bbe41fe869538ac35ffeea2: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 02:15:53.971558 18981 ts_tablet_manager.cc:1428] T 851019929b3b4bc69f3a7d616a4c2df1 P 09772a64befd467791220629b9d0561e: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260809 02:15:53.971827 18997 tablet_replica.cc:439] T 851019929b3b4bc69f3a7d616a4c2df1 P 09772a64befd467791220629b9d0561e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d22a2768bef4c7d9fc30d0311f732b0" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 36557 } } peers { permanent_uuid: "09772a64befd467791220629b9d0561e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45953 } } peers { permanent_uuid: "d678864a5bbe41fe869538ac35ffeea2" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45385 } } }
I20260809 02:15:53.971824 18986 raft_consensus.cc:348] T 851019929b3b4bc69f3a7d616a4c2df1 P 8d22a2768bef4c7d9fc30d0311f732b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d22a2768bef4c7d9fc30d0311f732b0" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 36557 } } peers { permanent_uuid: "09772a64befd467791220629b9d0561e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45953 } } peers { permanent_uuid: "d678864a5bbe41fe869538ac35ffeea2" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45385 } }
I20260809 02:15:53.971940 18986 raft_consensus.cc:374] T 851019929b3b4bc69f3a7d616a4c2df1 P 8d22a2768bef4c7d9fc30d0311f732b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:15:53.971951 18997 tablet_replica.cc:442] T 851019929b3b4bc69f3a7d616a4c2df1 P 09772a64befd467791220629b9d0561e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 02:15:53.971979 18986 raft_consensus.cc:729] T 851019929b3b4bc69f3a7d616a4c2df1 P 8d22a2768bef4c7d9fc30d0311f732b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d22a2768bef4c7d9fc30d0311f732b0, State: Initialized, Role: FOLLOWER
I20260809 02:15:53.972105 18986 consensus_queue.cc:260] T 851019929b3b4bc69f3a7d616a4c2df1 P 8d22a2768bef4c7d9fc30d0311f732b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d22a2768bef4c7d9fc30d0311f732b0" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 36557 } } peers { permanent_uuid: "09772a64befd467791220629b9d0561e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45953 } } peers { permanent_uuid: "d678864a5bbe41fe869538ac35ffeea2" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45385 } }
I20260809 02:15:53.972337 18995 tablet_replica.cc:439] T 851019929b3b4bc69f3a7d616a4c2df1 P 8d22a2768bef4c7d9fc30d0311f732b0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d22a2768bef4c7d9fc30d0311f732b0" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 36557 } } peers { permanent_uuid: "09772a64befd467791220629b9d0561e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45953 } } peers { permanent_uuid: "d678864a5bbe41fe869538ac35ffeea2" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45385 } } }
I20260809 02:15:53.972397 18986 ts_tablet_manager.cc:1428] T 851019929b3b4bc69f3a7d616a4c2df1 P 8d22a2768bef4c7d9fc30d0311f732b0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 02:15:53.972468 18995 tablet_replica.cc:442] T 851019929b3b4bc69f3a7d616a4c2df1 P 8d22a2768bef4c7d9fc30d0311f732b0: This TxnStatusTablet replica's current role is: FOLLOWER
W20260809 02:15:53.988210 18943 tablet.cc:2369] T 03c68b4fb78946d29fb07ebf4b9e1041 P d678864a5bbe41fe869538ac35ffeea2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:15:53.998086 18997 raft_consensus.cc:482] T 851019929b3b4bc69f3a7d616a4c2df1 P 09772a64befd467791220629b9d0561e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:15:53.998211 18997 raft_consensus.cc:504] T 851019929b3b4bc69f3a7d616a4c2df1 P 09772a64befd467791220629b9d0561e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d22a2768bef4c7d9fc30d0311f732b0" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 36557 } } peers { permanent_uuid: "09772a64befd467791220629b9d0561e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45953 } } peers { permanent_uuid: "d678864a5bbe41fe869538ac35ffeea2" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45385 } }
I20260809 02:15:53.999414 18997 leader_election.cc:288] T 851019929b3b4bc69f3a7d616a4c2df1 P 09772a64befd467791220629b9d0561e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8d22a2768bef4c7d9fc30d0311f732b0 (127.15.75.129:36557), d678864a5bbe41fe869538ac35ffeea2 (127.15.75.131:45385)
I20260809 02:15:54.004304 18971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:53.999679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.130:49681 (local address 127.15.75.129:36557)
0809 02:15:53.999805 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:53.999810 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:53.999824 (+ 14us) server_negotiation.cc:407] Connection header received
0809 02:15:53.999977 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:53.999980 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:54.000021 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:54.000118 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:15:54.000425 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:54.001254 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:54.003063 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:54.003672 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:54.003790 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:54.003842 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:54.004106 (+ 264us) server_negotiation.cc:299] Negotiation successful
0809 02:15:54.004177 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 02:15:54.004304 18979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:53.999737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.130:35663 (local address 127.15.75.131:45385)
0809 02:15:53.999860 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:53.999864 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:54.000099 (+ 235us) server_negotiation.cc:407] Connection header received
0809 02:15:54.000220 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:54.000223 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:54.000269 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:54.000327 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:15:54.000608 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:54.001322 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:54.003063 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:54.003668 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:54.003790 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:54.003842 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:54.004106 (+ 264us) server_negotiation.cc:299] Negotiation successful
0809 02:15:54.004177 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 02:15:54.004346 18592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:53.999562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.129:36557 (local address 127.15.75.130:49681)
0809 02:15:53.999732 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:53.999747 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:53.999832 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:54.000141 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:54.000147 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:54.000166 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:15:54.000280 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:54.000285 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:54.001364 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:54.001366 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:54.002932 (+ 1566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:54.002944 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:54.003792 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:54.003799 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:54.003934 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:54.003994 (+ 60us) client_negotiation.cc:769] Sending connection context
0809 02:15:54.004075 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 02:15:54.004176 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"mutex_wait_us":170}
I20260809 02:15:54.004350 19026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:53.999676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.131:45385 (local address 127.15.75.130:35663)
0809 02:15:53.999996 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:54.000007 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:54.000118 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:54.000368 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:54.000373 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:54.000396 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:15:54.000504 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:54.000509 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:54.001423 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:54.001426 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:54.002932 (+ 1506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:54.002944 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:54.003804 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:54.003807 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:54.003967 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:54.004003 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 02:15:54.004075 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 02:15:54.004176 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"mutex_wait_us":180,"thread_start_us":101,"threads_started":1}
I20260809 02:15:54.004926 18886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "851019929b3b4bc69f3a7d616a4c2df1" candidate_uuid: "09772a64befd467791220629b9d0561e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d678864a5bbe41fe869538ac35ffeea2" is_pre_election: true
I20260809 02:15:54.005050 18886 raft_consensus.cc:2454] T 851019929b3b4bc69f3a7d616a4c2df1 P d678864a5bbe41fe869538ac35ffeea2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09772a64befd467791220629b9d0561e in term 0.
I20260809 02:15:54.005085 18479 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "851019929b3b4bc69f3a7d616a4c2df1" candidate_uuid: "09772a64befd467791220629b9d0561e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d22a2768bef4c7d9fc30d0311f732b0" is_pre_election: true
I20260809 02:15:54.005306 18479 raft_consensus.cc:2454] T 851019929b3b4bc69f3a7d616a4c2df1 P 8d22a2768bef4c7d9fc30d0311f732b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09772a64befd467791220629b9d0561e in term 0.
I20260809 02:15:54.005446 18583 leader_election.cc:302] T 851019929b3b4bc69f3a7d616a4c2df1 P 09772a64befd467791220629b9d0561e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09772a64befd467791220629b9d0561e, d678864a5bbe41fe869538ac35ffeea2; no voters:
I20260809 02:15:54.005751 18997 raft_consensus.cc:2781] T 851019929b3b4bc69f3a7d616a4c2df1 P 09772a64befd467791220629b9d0561e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:15:54.005816 18997 raft_consensus.cc:482] T 851019929b3b4bc69f3a7d616a4c2df1 P 09772a64befd467791220629b9d0561e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:15:54.005873 18997 raft_consensus.cc:3037] T 851019929b3b4bc69f3a7d616a4c2df1 P 09772a64befd467791220629b9d0561e [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:15:54.006986 18997 raft_consensus.cc:504] T 851019929b3b4bc69f3a7d616a4c2df1 P 09772a64befd467791220629b9d0561e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d22a2768bef4c7d9fc30d0311f732b0" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 36557 } } peers { permanent_uuid: "09772a64befd467791220629b9d0561e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45953 } } peers { permanent_uuid: "d678864a5bbe41fe869538ac35ffeea2" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45385 } }
I20260809 02:15:54.007340 18997 leader_election.cc:288] T 851019929b3b4bc69f3a7d616a4c2df1 P 09772a64befd467791220629b9d0561e [CANDIDATE]: Term 1 election: Requested vote from peers 8d22a2768bef4c7d9fc30d0311f732b0 (127.15.75.129:36557), d678864a5bbe41fe869538ac35ffeea2 (127.15.75.131:45385)
I20260809 02:15:54.007704 18886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "851019929b3b4bc69f3a7d616a4c2df1" candidate_uuid: "09772a64befd467791220629b9d0561e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d678864a5bbe41fe869538ac35ffeea2"
I20260809 02:15:54.007732 18479 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "851019929b3b4bc69f3a7d616a4c2df1" candidate_uuid: "09772a64befd467791220629b9d0561e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d22a2768bef4c7d9fc30d0311f732b0"
I20260809 02:15:54.007862 18479 raft_consensus.cc:3037] T 851019929b3b4bc69f3a7d616a4c2df1 P 8d22a2768bef4c7d9fc30d0311f732b0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:15:54.007860 18886 raft_consensus.cc:3037] T 851019929b3b4bc69f3a7d616a4c2df1 P d678864a5bbe41fe869538ac35ffeea2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:15:54.008857 18479 raft_consensus.cc:2454] T 851019929b3b4bc69f3a7d616a4c2df1 P 8d22a2768bef4c7d9fc30d0311f732b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09772a64befd467791220629b9d0561e in term 1.
I20260809 02:15:54.008910 18886 raft_consensus.cc:2454] T 851019929b3b4bc69f3a7d616a4c2df1 P d678864a5bbe41fe869538ac35ffeea2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09772a64befd467791220629b9d0561e in term 1.
I20260809 02:15:54.009223 18583 leader_election.cc:302] T 851019929b3b4bc69f3a7d616a4c2df1 P 09772a64befd467791220629b9d0561e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09772a64befd467791220629b9d0561e, d678864a5bbe41fe869538ac35ffeea2; no voters:
I20260809 02:15:54.009415 18997 raft_consensus.cc:2781] T 851019929b3b4bc69f3a7d616a4c2df1 P 09772a64befd467791220629b9d0561e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:15:54.009627 18997 raft_consensus.cc:686] T 851019929b3b4bc69f3a7d616a4c2df1 P 09772a64befd467791220629b9d0561e [term 1 LEADER]: Becoming Leader. State: Replica: 09772a64befd467791220629b9d0561e, State: Running, Role: LEADER
I20260809 02:15:54.009976 18997 consensus_queue.cc:237] T 851019929b3b4bc69f3a7d616a4c2df1 P 09772a64befd467791220629b9d0561e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d22a2768bef4c7d9fc30d0311f732b0" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 36557 } } peers { permanent_uuid: "09772a64befd467791220629b9d0561e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45953 } } peers { permanent_uuid: "d678864a5bbe41fe869538ac35ffeea2" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45385 } }
I20260809 02:15:54.012310 19027 tablet_replica.cc:439] T 851019929b3b4bc69f3a7d616a4c2df1 P 09772a64befd467791220629b9d0561e: TxnStatusTablet state changed. Reason: New leader 09772a64befd467791220629b9d0561e. Latest consensus state: current_term: 1 leader_uuid: "09772a64befd467791220629b9d0561e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d22a2768bef4c7d9fc30d0311f732b0" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 36557 } } peers { permanent_uuid: "09772a64befd467791220629b9d0561e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45953 } } peers { permanent_uuid: "d678864a5bbe41fe869538ac35ffeea2" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45385 } } }
I20260809 02:15:54.012419 19027 tablet_replica.cc:442] T 851019929b3b4bc69f3a7d616a4c2df1 P 09772a64befd467791220629b9d0561e: This TxnStatusTablet replica's current role is: LEADER
I20260809 02:15:54.012740 19029 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 02:15:54.012961 19029 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 02:15:54.013413 18245 catalog_manager.cc:5495] T 851019929b3b4bc69f3a7d616a4c2df1 P 09772a64befd467791220629b9d0561e reported cstate change: term changed from 0 to 1, leader changed from <none> to 09772a64befd467791220629b9d0561e (127.15.75.130). New cstate: current_term: 1 leader_uuid: "09772a64befd467791220629b9d0561e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d22a2768bef4c7d9fc30d0311f732b0" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 36557 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09772a64befd467791220629b9d0561e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d678864a5bbe41fe869538ac35ffeea2" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45385 } health_report { overall_health: UNKNOWN } } }
I20260809 02:15:54.026099 18978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:54.022263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48480 (local address 127.15.75.130:45953)
0809 02:15:54.022393 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:54.022397 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:54.022413 (+ 16us) server_negotiation.cc:407] Connection header received
0809 02:15:54.022551 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:54.022554 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:54.022629 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:54.022755 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:15:54.022987 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:54.023769 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:54.024653 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:54.024926 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:54.025038 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:54.025330 (+ 292us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:15:54.025709 (+ 379us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:15:54.025835 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:54.025929 (+ 94us) server_negotiation.cc:299] Negotiation successful
0809 02:15:54.025976 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 02:15:54.026099 19022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:54.022177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.130:45953 (local address 127.0.0.1:48480)
0809 02:15:54.022318 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:54.022333 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:54.022425 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:54.022767 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:54.022770 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:54.022775 (+ 5us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 02:15:54.022882 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:54.022886 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:54.023884 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:54.023886 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:54.024537 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:54.024546 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:54.025040 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:54.025044 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:54.025138 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:54.025179 (+ 41us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 02:15:54.025816 (+ 637us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 02:15:54.025834 (+ 18us) client_negotiation.cc:769] Sending connection context
0809 02:15:54.025913 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 02:15:54.025990 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260809 02:15:54.031337 18886 raft_consensus.cc:1261] T 851019929b3b4bc69f3a7d616a4c2df1 P d678864a5bbe41fe869538ac35ffeea2 [term 1 FOLLOWER]: Refusing update from remote peer 09772a64befd467791220629b9d0561e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:15:54.031406 18479 raft_consensus.cc:1261] T 851019929b3b4bc69f3a7d616a4c2df1 P 8d22a2768bef4c7d9fc30d0311f732b0 [term 1 FOLLOWER]: Refusing update from remote peer 09772a64befd467791220629b9d0561e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260809 02:15:54.031816 18746 tablet.cc:2369] T 03c68b4fb78946d29fb07ebf4b9e1041 P 09772a64befd467791220629b9d0561e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:15:54.031962 19027 consensus_queue.cc:1035] T 851019929b3b4bc69f3a7d616a4c2df1 P 09772a64befd467791220629b9d0561e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d22a2768bef4c7d9fc30d0311f732b0" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 36557 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:15:54.032610 19027 consensus_queue.cc:1035] T 851019929b3b4bc69f3a7d616a4c2df1 P 09772a64befd467791220629b9d0561e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d678864a5bbe41fe869538ac35ffeea2" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:15:54.035658 18995 tablet_replica.cc:439] T 851019929b3b4bc69f3a7d616a4c2df1 P 8d22a2768bef4c7d9fc30d0311f732b0: TxnStatusTablet state changed. Reason: New leader 09772a64befd467791220629b9d0561e. Latest consensus state: current_term: 1 leader_uuid: "09772a64befd467791220629b9d0561e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d22a2768bef4c7d9fc30d0311f732b0" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 36557 } } peers { permanent_uuid: "09772a64befd467791220629b9d0561e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45953 } } peers { permanent_uuid: "d678864a5bbe41fe869538ac35ffeea2" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45385 } } }
I20260809 02:15:54.035784 18995 tablet_replica.cc:442] T 851019929b3b4bc69f3a7d616a4c2df1 P 8d22a2768bef4c7d9fc30d0311f732b0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 02:15:54.036616 18994 tablet_replica.cc:439] T 851019929b3b4bc69f3a7d616a4c2df1 P d678864a5bbe41fe869538ac35ffeea2: TxnStatusTablet state changed. Reason: New leader 09772a64befd467791220629b9d0561e. Latest consensus state: current_term: 1 leader_uuid: "09772a64befd467791220629b9d0561e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d22a2768bef4c7d9fc30d0311f732b0" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 36557 } } peers { permanent_uuid: "09772a64befd467791220629b9d0561e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45953 } } peers { permanent_uuid: "d678864a5bbe41fe869538ac35ffeea2" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45385 } } }
I20260809 02:15:54.036777 18994 tablet_replica.cc:442] T 851019929b3b4bc69f3a7d616a4c2df1 P d678864a5bbe41fe869538ac35ffeea2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 02:15:54.039599 19027 tablet_replica.cc:439] T 851019929b3b4bc69f3a7d616a4c2df1 P 09772a64befd467791220629b9d0561e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "09772a64befd467791220629b9d0561e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d22a2768bef4c7d9fc30d0311f732b0" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 36557 } } peers { permanent_uuid: "09772a64befd467791220629b9d0561e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45953 } } peers { permanent_uuid: "d678864a5bbe41fe869538ac35ffeea2" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45385 } } }
I20260809 02:15:54.039710 19027 tablet_replica.cc:442] T 851019929b3b4bc69f3a7d616a4c2df1 P 09772a64befd467791220629b9d0561e: This TxnStatusTablet replica's current role is: LEADER
I20260809 02:15:54.041436 19037 mvcc.cc:205] Tried to move back new op lower bound from 7316446224506007552 to 7316446224427610112. Current Snapshot: MvccSnapshot[applied={T|T < 7316446224506007552}]
I20260809 02:15:54.041765 18994 tablet_replica.cc:439] T 851019929b3b4bc69f3a7d616a4c2df1 P d678864a5bbe41fe869538ac35ffeea2: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "09772a64befd467791220629b9d0561e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d22a2768bef4c7d9fc30d0311f732b0" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 36557 } } peers { permanent_uuid: "09772a64befd467791220629b9d0561e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45953 } } peers { permanent_uuid: "d678864a5bbe41fe869538ac35ffeea2" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45385 } } }
I20260809 02:15:54.041869 18994 tablet_replica.cc:442] T 851019929b3b4bc69f3a7d616a4c2df1 P d678864a5bbe41fe869538ac35ffeea2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 02:15:54.045133 18997 tablet_replica.cc:439] T 851019929b3b4bc69f3a7d616a4c2df1 P 09772a64befd467791220629b9d0561e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "09772a64befd467791220629b9d0561e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d22a2768bef4c7d9fc30d0311f732b0" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 36557 } } peers { permanent_uuid: "09772a64befd467791220629b9d0561e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45953 } } peers { permanent_uuid: "d678864a5bbe41fe869538ac35ffeea2" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45385 } } }
I20260809 02:15:54.045246 18997 tablet_replica.cc:442] T 851019929b3b4bc69f3a7d616a4c2df1 P 09772a64befd467791220629b9d0561e: This TxnStatusTablet replica's current role is: LEADER
I20260809 02:15:54.046208 18995 tablet_replica.cc:439] T 851019929b3b4bc69f3a7d616a4c2df1 P 8d22a2768bef4c7d9fc30d0311f732b0: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "09772a64befd467791220629b9d0561e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d22a2768bef4c7d9fc30d0311f732b0" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 36557 } } peers { permanent_uuid: "09772a64befd467791220629b9d0561e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45953 } } peers { permanent_uuid: "d678864a5bbe41fe869538ac35ffeea2" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45385 } } }
I20260809 02:15:54.046324 18995 tablet_replica.cc:442] T 851019929b3b4bc69f3a7d616a4c2df1 P 8d22a2768bef4c7d9fc30d0311f732b0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 02:15:54.048277 19035 mvcc.cc:205] Tried to move back new op lower bound from 7316446224506007552 to 7316446224427610112. Current Snapshot: MvccSnapshot[applied={T|T < 7316446224506007552 or (T in {7316446224506007552})}]
I20260809 02:15:54.054322 18971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:54.048617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53640 (local address 127.15.75.129:36557)
0809 02:15:54.048747 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:54.048752 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:54.049592 (+ 840us) server_negotiation.cc:407] Connection header received
0809 02:15:54.049773 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:54.049778 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:54.049841 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:54.049933 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:15:54.050512 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:54.051269 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:54.052461 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:54.052829 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:54.052922 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:54.053296 (+ 374us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:15:54.053763 (+ 467us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:15:54.053863 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:54.054147 (+ 284us) server_negotiation.cc:299] Negotiation successful
0809 02:15:54.054209 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
W20260809 02:15:54.063275 18547 tablet.cc:2369] T 03c68b4fb78946d29fb07ebf4b9e1041 P 8d22a2768bef4c7d9fc30d0311f732b0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:15:54.067968 19048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:54.064377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.130:45953 (local address 127.15.75.129:47159)
0809 02:15:54.064660 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:54.064672 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:54.064834 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:54.065070 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:54.065072 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:54.065077 (+ 5us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 02:15:54.065177 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:54.065181 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:54.066139 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:54.066142 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:54.066747 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:54.066755 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:54.067308 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:54.067311 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:54.067379 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:54.067421 (+ 42us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 02:15:54.067727 (+ 306us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 02:15:54.067731 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:15:54.067779 (+ 48us) client_negotiation.cc:240] Negotiation successful
0809 02:15:54.067837 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":90,"threads_started":1}
I20260809 02:15:54.067967 18978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:54.064474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.129:47159 (local address 127.15.75.130:45953)
0809 02:15:54.064676 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:54.064709 (+ 33us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:54.064782 (+ 73us) server_negotiation.cc:407] Connection header received
0809 02:15:54.064928 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:54.064931 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:54.064982 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:54.065066 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:15:54.065270 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:54.066031 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:54.066858 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:54.067196 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:54.067307 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:54.067564 (+ 257us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:15:54.067642 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:15:54.067723 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:54.067791 (+ 68us) server_negotiation.cc:299] Negotiation successful
0809 02:15:54.067840 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260809 02:15:54.074853 18678 raft_consensus.cc:1261] T 03c68b4fb78946d29fb07ebf4b9e1041 P 09772a64befd467791220629b9d0561e [term 1 FOLLOWER]: Refusing update from remote peer 8d22a2768bef4c7d9fc30d0311f732b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:15:54.075304 18886 raft_consensus.cc:1261] T 03c68b4fb78946d29fb07ebf4b9e1041 P d678864a5bbe41fe869538ac35ffeea2 [term 1 FOLLOWER]: Refusing update from remote peer 8d22a2768bef4c7d9fc30d0311f732b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:15:54.075416 19005 consensus_queue.cc:1035] T 03c68b4fb78946d29fb07ebf4b9e1041 P 8d22a2768bef4c7d9fc30d0311f732b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09772a64befd467791220629b9d0561e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:15:54.075660 18995 consensus_queue.cc:1035] T 03c68b4fb78946d29fb07ebf4b9e1041 P 8d22a2768bef4c7d9fc30d0311f732b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d678864a5bbe41fe869538ac35ffeea2" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:15:54.081703 19034 mvcc.cc:205] Tried to move back new op lower bound from 7316446224685711360 to 7316446223965089792. Current Snapshot: MvccSnapshot[applied={T|T < 7316446224685711360}]
I20260809 02:15:54.098975 18971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:54.094121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.130:33445 (local address 127.15.75.129:36557)
0809 02:15:54.094256 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:54.094262 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:54.094511 (+ 249us) server_negotiation.cc:407] Connection header received
0809 02:15:54.094609 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:54.094612 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:54.094658 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:54.094743 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:15:54.095585 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:54.096398 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:54.097710 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:54.098023 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:54.098143 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:54.098501 (+ 358us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:15:54.098573 (+ 72us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:15:54.098636 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:54.098764 (+ 128us) server_negotiation.cc:299] Negotiation successful
0809 02:15:54.098835 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 02:15:54.099014 19056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:54.094053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.129:36557 (local address 127.15.75.130:33445)
0809 02:15:54.094426 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:54.094441 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:54.094535 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:54.094762 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:54.094764 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:54.094771 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 02:15:54.095452 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:54.095468 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:54.096536 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:54.096539 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:54.097582 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:54.097594 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:54.098170 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:54.098175 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:54.098309 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:54.098362 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 02:15:54.098665 (+ 303us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 02:15:54.098672 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 02:15:54.098741 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 02:15:54.098820 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":124,"threads_started":1}
I20260809 02:15:54.128162 18245 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43022:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:15:54.128368 18245 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43022:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:15:54.131119 18245 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e222f3caa8a145b9ad29749cb134b63c: Sending DeleteTablet for 3 replicas of tablet 03c68b4fb78946d29fb07ebf4b9e1041
I20260809 02:15:54.132014 18447 tablet_service.cc:1514] Processing DeleteTablet for tablet 03c68b4fb78946d29fb07ebf4b9e1041 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:15:54 UTC) from {username='kudu', principal='kudu/127.15.75.190@KRBTEST.COM'} at 127.0.0.1:53630
I20260809 02:15:54.132010 18857 tablet_service.cc:1514] Processing DeleteTablet for tablet 03c68b4fb78946d29fb07ebf4b9e1041 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:15:54 UTC) from {username='kudu', principal='kudu/127.15.75.190@KRBTEST.COM'} at 127.0.0.1:55542
I20260809 02:15:54.132609 18648 tablet_service.cc:1514] Processing DeleteTablet for tablet 03c68b4fb78946d29fb07ebf4b9e1041 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:15:54 UTC) from {username='kudu', principal='kudu/127.15.75.190@KRBTEST.COM'} at 127.0.0.1:48474
I20260809 02:15:54.133178 19060 tablet_replica.cc:330] T 03c68b4fb78946d29fb07ebf4b9e1041 P 8d22a2768bef4c7d9fc30d0311f732b0: stopping tablet replica
I20260809 02:15:54.133260 18245 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43022:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 02:15:54.133395 19062 tablet_replica.cc:330] T 03c68b4fb78946d29fb07ebf4b9e1041 P 09772a64befd467791220629b9d0561e: stopping tablet replica
W20260809 02:15:54.133651 18245 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:15:54.134176 19062 raft_consensus.cc:2229] T 03c68b4fb78946d29fb07ebf4b9e1041 P 09772a64befd467791220629b9d0561e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:15:54.134392 19060 raft_consensus.cc:2229] T 03c68b4fb78946d29fb07ebf4b9e1041 P 8d22a2768bef4c7d9fc30d0311f732b0 [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:15:54.134630 19062 raft_consensus.cc:2258] T 03c68b4fb78946d29fb07ebf4b9e1041 P 09772a64befd467791220629b9d0561e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:15:54.134918 19061 tablet_replica.cc:330] T 03c68b4fb78946d29fb07ebf4b9e1041 P d678864a5bbe41fe869538ac35ffeea2: stopping tablet replica
I20260809 02:15:54.135190 19061 raft_consensus.cc:2229] T 03c68b4fb78946d29fb07ebf4b9e1041 P d678864a5bbe41fe869538ac35ffeea2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:15:54.135430 19061 raft_consensus.cc:2258] T 03c68b4fb78946d29fb07ebf4b9e1041 P d678864a5bbe41fe869538ac35ffeea2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:15:54.136240 19060 raft_consensus.cc:2258] T 03c68b4fb78946d29fb07ebf4b9e1041 P 8d22a2768bef4c7d9fc30d0311f732b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:15:54.139009 18648 tablet_service.cc:1467] Processing CreateTablet for tablet 1993a70bf1dd40c3b99039201b62ec5b (DEFAULT_TABLE table=test-table [id=147c9a6457ea4a4cbf9bbddbc62a9c6a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:15:54.139040 18857 tablet_service.cc:1467] Processing CreateTablet for tablet 1993a70bf1dd40c3b99039201b62ec5b (DEFAULT_TABLE table=test-table [id=147c9a6457ea4a4cbf9bbddbc62a9c6a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:15:54.139324 18648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1993a70bf1dd40c3b99039201b62ec5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:15:54.139326 18857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1993a70bf1dd40c3b99039201b62ec5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:15:54.141840 18987 tablet_bootstrap.cc:492] T 1993a70bf1dd40c3b99039201b62ec5b P d678864a5bbe41fe869538ac35ffeea2: Bootstrap starting.
I20260809 02:15:54.142796 18987 tablet_bootstrap.cc:654] T 1993a70bf1dd40c3b99039201b62ec5b P d678864a5bbe41fe869538ac35ffeea2: Neither blocks nor log segments found. Creating new log.
I20260809 02:15:54.138567 18447 tablet_service.cc:1467] Processing CreateTablet for tablet 1993a70bf1dd40c3b99039201b62ec5b (DEFAULT_TABLE table=test-table [id=147c9a6457ea4a4cbf9bbddbc62a9c6a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:15:54.143105 18447 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1993a70bf1dd40c3b99039201b62ec5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:15:54.143427 18981 tablet_bootstrap.cc:492] T 1993a70bf1dd40c3b99039201b62ec5b P 09772a64befd467791220629b9d0561e: Bootstrap starting.
I20260809 02:15:54.144362 18981 tablet_bootstrap.cc:654] T 1993a70bf1dd40c3b99039201b62ec5b P 09772a64befd467791220629b9d0561e: Neither blocks nor log segments found. Creating new log.
I20260809 02:15:54.145964 18986 tablet_bootstrap.cc:492] T 1993a70bf1dd40c3b99039201b62ec5b P 8d22a2768bef4c7d9fc30d0311f732b0: Bootstrap starting.
I20260809 02:15:54.146512 18981 tablet_bootstrap.cc:492] T 1993a70bf1dd40c3b99039201b62ec5b P 09772a64befd467791220629b9d0561e: No bootstrap required, opened a new log
I20260809 02:15:54.146584 18981 ts_tablet_manager.cc:1397] T 1993a70bf1dd40c3b99039201b62ec5b P 09772a64befd467791220629b9d0561e: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260809 02:15:54.146945 18986 tablet_bootstrap.cc:654] T 1993a70bf1dd40c3b99039201b62ec5b P 8d22a2768bef4c7d9fc30d0311f732b0: Neither blocks nor log segments found. Creating new log.
I20260809 02:15:54.147054 18981 raft_consensus.cc:348] T 1993a70bf1dd40c3b99039201b62ec5b P 09772a64befd467791220629b9d0561e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d22a2768bef4c7d9fc30d0311f732b0" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 36557 } } peers { permanent_uuid: "09772a64befd467791220629b9d0561e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45953 } } peers { permanent_uuid: "d678864a5bbe41fe869538ac35ffeea2" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45385 } }
I20260809 02:15:54.147176 18981 raft_consensus.cc:374] T 1993a70bf1dd40c3b99039201b62ec5b P 09772a64befd467791220629b9d0561e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:15:54.147214 18981 raft_consensus.cc:729] T 1993a70bf1dd40c3b99039201b62ec5b P 09772a64befd467791220629b9d0561e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09772a64befd467791220629b9d0561e, State: Initialized, Role: FOLLOWER
I20260809 02:15:54.147346 18981 consensus_queue.cc:260] T 1993a70bf1dd40c3b99039201b62ec5b P 09772a64befd467791220629b9d0561e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d22a2768bef4c7d9fc30d0311f732b0" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 36557 } } peers { permanent_uuid: "09772a64befd467791220629b9d0561e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45953 } } peers { permanent_uuid: "d678864a5bbe41fe869538ac35ffeea2" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45385 } }
I20260809 02:15:54.147598 18981 ts_tablet_manager.cc:1428] T 1993a70bf1dd40c3b99039201b62ec5b P 09772a64befd467791220629b9d0561e: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260809 02:15:54.149370 18986 tablet_bootstrap.cc:492] T 1993a70bf1dd40c3b99039201b62ec5b P 8d22a2768bef4c7d9fc30d0311f732b0: No bootstrap required, opened a new log
I20260809 02:15:54.149454 18986 ts_tablet_manager.cc:1397] T 1993a70bf1dd40c3b99039201b62ec5b P 8d22a2768bef4c7d9fc30d0311f732b0: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260809 02:15:54.149885 18986 raft_consensus.cc:348] T 1993a70bf1dd40c3b99039201b62ec5b P 8d22a2768bef4c7d9fc30d0311f732b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d22a2768bef4c7d9fc30d0311f732b0" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 36557 } } peers { permanent_uuid: "09772a64befd467791220629b9d0561e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45953 } } peers { permanent_uuid: "d678864a5bbe41fe869538ac35ffeea2" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45385 } }
I20260809 02:15:54.149995 18986 raft_consensus.cc:374] T 1993a70bf1dd40c3b99039201b62ec5b P 8d22a2768bef4c7d9fc30d0311f732b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:15:54.150024 18986 raft_consensus.cc:729] T 1993a70bf1dd40c3b99039201b62ec5b P 8d22a2768bef4c7d9fc30d0311f732b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d22a2768bef4c7d9fc30d0311f732b0, State: Initialized, Role: FOLLOWER
I20260809 02:15:54.150132 18986 consensus_queue.cc:260] T 1993a70bf1dd40c3b99039201b62ec5b P 8d22a2768bef4c7d9fc30d0311f732b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d22a2768bef4c7d9fc30d0311f732b0" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 36557 } } peers { permanent_uuid: "09772a64befd467791220629b9d0561e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45953 } } peers { permanent_uuid: "d678864a5bbe41fe869538ac35ffeea2" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45385 } }
I20260809 02:15:54.150372 18986 ts_tablet_manager.cc:1428] T 1993a70bf1dd40c3b99039201b62ec5b P 8d22a2768bef4c7d9fc30d0311f732b0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 02:15:54.151077 18987 tablet_bootstrap.cc:492] T 1993a70bf1dd40c3b99039201b62ec5b P d678864a5bbe41fe869538ac35ffeea2: No bootstrap required, opened a new log
I20260809 02:15:54.151170 18987 ts_tablet_manager.cc:1397] T 1993a70bf1dd40c3b99039201b62ec5b P d678864a5bbe41fe869538ac35ffeea2: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.002s
I20260809 02:15:54.151656 18987 raft_consensus.cc:348] T 1993a70bf1dd40c3b99039201b62ec5b P d678864a5bbe41fe869538ac35ffeea2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d22a2768bef4c7d9fc30d0311f732b0" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 36557 } } peers { permanent_uuid: "09772a64befd467791220629b9d0561e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45953 } } peers { permanent_uuid: "d678864a5bbe41fe869538ac35ffeea2" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45385 } }
I20260809 02:15:54.151809 18987 raft_consensus.cc:374] T 1993a70bf1dd40c3b99039201b62ec5b P d678864a5bbe41fe869538ac35ffeea2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:15:54.151851 18987 raft_consensus.cc:729] T 1993a70bf1dd40c3b99039201b62ec5b P d678864a5bbe41fe869538ac35ffeea2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d678864a5bbe41fe869538ac35ffeea2, State: Initialized, Role: FOLLOWER
I20260809 02:15:54.151993 18987 consensus_queue.cc:260] T 1993a70bf1dd40c3b99039201b62ec5b P d678864a5bbe41fe869538ac35ffeea2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d22a2768bef4c7d9fc30d0311f732b0" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 36557 } } peers { permanent_uuid: "09772a64befd467791220629b9d0561e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45953 } } peers { permanent_uuid: "d678864a5bbe41fe869538ac35ffeea2" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45385 } }
I20260809 02:15:54.152273 18987 ts_tablet_manager.cc:1428] T 1993a70bf1dd40c3b99039201b62ec5b P d678864a5bbe41fe869538ac35ffeea2: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260809 02:15:54.152661 19062 ts_tablet_manager.cc:1905] T 03c68b4fb78946d29fb07ebf4b9e1041 P 09772a64befd467791220629b9d0561e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:15:54.155701 19060 ts_tablet_manager.cc:1905] T 03c68b4fb78946d29fb07ebf4b9e1041 P 8d22a2768bef4c7d9fc30d0311f732b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:15:54.157187 19061 ts_tablet_manager.cc:1905] T 03c68b4fb78946d29fb07ebf4b9e1041 P d678864a5bbe41fe869538ac35ffeea2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:15:54.159209 19060 ts_tablet_manager.cc:1918] T 03c68b4fb78946d29fb07ebf4b9e1041 P 8d22a2768bef4c7d9fc30d0311f732b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 02:15:54.159322 19060 log.cc:1196] T 03c68b4fb78946d29fb07ebf4b9e1041 P 8d22a2768bef4c7d9fc30d0311f732b0: Deleting WAL directory at /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/minicluster-data/ts-0/wal/wals/03c68b4fb78946d29fb07ebf4b9e1041
I20260809 02:15:54.159662 19060 ts_tablet_manager.cc:1939] T 03c68b4fb78946d29fb07ebf4b9e1041 P 8d22a2768bef4c7d9fc30d0311f732b0: Deleting consensus metadata
I20260809 02:15:54.160835 18234 catalog_manager.cc:4841] TS 8d22a2768bef4c7d9fc30d0311f732b0 (127.15.75.129:36557): tablet 03c68b4fb78946d29fb07ebf4b9e1041 (table test-table [id=f4c79ab4e364470893686ae03f6c07a1]) successfully deleted
I20260809 02:15:54.162616 19062 ts_tablet_manager.cc:1918] T 03c68b4fb78946d29fb07ebf4b9e1041 P 09772a64befd467791220629b9d0561e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 02:15:54.163414 19062 log.cc:1196] T 03c68b4fb78946d29fb07ebf4b9e1041 P 09772a64befd467791220629b9d0561e: Deleting WAL directory at /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/minicluster-data/ts-1/wal/wals/03c68b4fb78946d29fb07ebf4b9e1041
I20260809 02:15:54.163800 19062 ts_tablet_manager.cc:1939] T 03c68b4fb78946d29fb07ebf4b9e1041 P 09772a64befd467791220629b9d0561e: Deleting consensus metadata
I20260809 02:15:54.164091 19061 ts_tablet_manager.cc:1918] T 03c68b4fb78946d29fb07ebf4b9e1041 P d678864a5bbe41fe869538ac35ffeea2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 02:15:54.164233 19061 log.cc:1196] T 03c68b4fb78946d29fb07ebf4b9e1041 P d678864a5bbe41fe869538ac35ffeea2: Deleting WAL directory at /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/minicluster-data/ts-2/wal/wals/03c68b4fb78946d29fb07ebf4b9e1041
I20260809 02:15:54.164595 19061 ts_tablet_manager.cc:1939] T 03c68b4fb78946d29fb07ebf4b9e1041 P d678864a5bbe41fe869538ac35ffeea2: Deleting consensus metadata
I20260809 02:15:54.165016 18236 catalog_manager.cc:4841] TS 09772a64befd467791220629b9d0561e (127.15.75.130:45953): tablet 03c68b4fb78946d29fb07ebf4b9e1041 (table test-table [id=f4c79ab4e364470893686ae03f6c07a1]) successfully deleted
I20260809 02:15:54.165874 18229 catalog_manager.cc:4841] TS d678864a5bbe41fe869538ac35ffeea2 (127.15.75.131:45385): tablet 03c68b4fb78946d29fb07ebf4b9e1041 (table test-table [id=f4c79ab4e364470893686ae03f6c07a1]) successfully deleted
I20260809 02:15:54.357453 18995 raft_consensus.cc:482] T 1993a70bf1dd40c3b99039201b62ec5b P 8d22a2768bef4c7d9fc30d0311f732b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:15:54.357627 18995 raft_consensus.cc:504] T 1993a70bf1dd40c3b99039201b62ec5b P 8d22a2768bef4c7d9fc30d0311f732b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d22a2768bef4c7d9fc30d0311f732b0" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 36557 } } peers { permanent_uuid: "09772a64befd467791220629b9d0561e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45953 } } peers { permanent_uuid: "d678864a5bbe41fe869538ac35ffeea2" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45385 } }
I20260809 02:15:54.358095 18995 leader_election.cc:288] T 1993a70bf1dd40c3b99039201b62ec5b P 8d22a2768bef4c7d9fc30d0311f732b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 09772a64befd467791220629b9d0561e (127.15.75.130:45953), d678864a5bbe41fe869538ac35ffeea2 (127.15.75.131:45385)
I20260809 02:15:54.358621 18886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1993a70bf1dd40c3b99039201b62ec5b" candidate_uuid: "8d22a2768bef4c7d9fc30d0311f732b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d678864a5bbe41fe869538ac35ffeea2" is_pre_election: true
I20260809 02:15:54.358644 18677 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1993a70bf1dd40c3b99039201b62ec5b" candidate_uuid: "8d22a2768bef4c7d9fc30d0311f732b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09772a64befd467791220629b9d0561e" is_pre_election: true
I20260809 02:15:54.358762 18886 raft_consensus.cc:2454] T 1993a70bf1dd40c3b99039201b62ec5b P d678864a5bbe41fe869538ac35ffeea2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d22a2768bef4c7d9fc30d0311f732b0 in term 0.
I20260809 02:15:54.358872 18677 raft_consensus.cc:2454] T 1993a70bf1dd40c3b99039201b62ec5b P 09772a64befd467791220629b9d0561e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d22a2768bef4c7d9fc30d0311f732b0 in term 0.
I20260809 02:15:54.359141 18374 leader_election.cc:302] T 1993a70bf1dd40c3b99039201b62ec5b P 8d22a2768bef4c7d9fc30d0311f732b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d22a2768bef4c7d9fc30d0311f732b0, d678864a5bbe41fe869538ac35ffeea2; no voters:
I20260809 02:15:54.359313 18995 raft_consensus.cc:2781] T 1993a70bf1dd40c3b99039201b62ec5b P 8d22a2768bef4c7d9fc30d0311f732b0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:15:54.359375 18995 raft_consensus.cc:482] T 1993a70bf1dd40c3b99039201b62ec5b P 8d22a2768bef4c7d9fc30d0311f732b0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:15:54.359450 18995 raft_consensus.cc:3037] T 1993a70bf1dd40c3b99039201b62ec5b P 8d22a2768bef4c7d9fc30d0311f732b0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:15:54.360447 18995 raft_consensus.cc:504] T 1993a70bf1dd40c3b99039201b62ec5b P 8d22a2768bef4c7d9fc30d0311f732b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d22a2768bef4c7d9fc30d0311f732b0" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 36557 } } peers { permanent_uuid: "09772a64befd467791220629b9d0561e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45953 } } peers { permanent_uuid: "d678864a5bbe41fe869538ac35ffeea2" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45385 } }
I20260809 02:15:54.360841 18995 leader_election.cc:288] T 1993a70bf1dd40c3b99039201b62ec5b P 8d22a2768bef4c7d9fc30d0311f732b0 [CANDIDATE]: Term 1 election: Requested vote from peers 09772a64befd467791220629b9d0561e (127.15.75.130:45953), d678864a5bbe41fe869538ac35ffeea2 (127.15.75.131:45385)
I20260809 02:15:54.361163 18677 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1993a70bf1dd40c3b99039201b62ec5b" candidate_uuid: "8d22a2768bef4c7d9fc30d0311f732b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09772a64befd467791220629b9d0561e"
I20260809 02:15:54.361186 18886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1993a70bf1dd40c3b99039201b62ec5b" candidate_uuid: "8d22a2768bef4c7d9fc30d0311f732b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d678864a5bbe41fe869538ac35ffeea2"
I20260809 02:15:54.361289 18886 raft_consensus.cc:3037] T 1993a70bf1dd40c3b99039201b62ec5b P d678864a5bbe41fe869538ac35ffeea2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:15:54.361289 18677 raft_consensus.cc:3037] T 1993a70bf1dd40c3b99039201b62ec5b P 09772a64befd467791220629b9d0561e [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:15:54.362432 18886 raft_consensus.cc:2454] T 1993a70bf1dd40c3b99039201b62ec5b P d678864a5bbe41fe869538ac35ffeea2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d22a2768bef4c7d9fc30d0311f732b0 in term 1.
I20260809 02:15:54.362493 18677 raft_consensus.cc:2454] T 1993a70bf1dd40c3b99039201b62ec5b P 09772a64befd467791220629b9d0561e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d22a2768bef4c7d9fc30d0311f732b0 in term 1.
I20260809 02:15:54.362746 18377 leader_election.cc:302] T 1993a70bf1dd40c3b99039201b62ec5b P 8d22a2768bef4c7d9fc30d0311f732b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 09772a64befd467791220629b9d0561e, 8d22a2768bef4c7d9fc30d0311f732b0, d678864a5bbe41fe869538ac35ffeea2; no voters:
I20260809 02:15:54.362949 18995 raft_consensus.cc:2781] T 1993a70bf1dd40c3b99039201b62ec5b P 8d22a2768bef4c7d9fc30d0311f732b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:15:54.363065 18995 raft_consensus.cc:686] T 1993a70bf1dd40c3b99039201b62ec5b P 8d22a2768bef4c7d9fc30d0311f732b0 [term 1 LEADER]: Becoming Leader. State: Replica: 8d22a2768bef4c7d9fc30d0311f732b0, State: Running, Role: LEADER
I20260809 02:15:54.363212 18995 consensus_queue.cc:237] T 1993a70bf1dd40c3b99039201b62ec5b P 8d22a2768bef4c7d9fc30d0311f732b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d22a2768bef4c7d9fc30d0311f732b0" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 36557 } } peers { permanent_uuid: "09772a64befd467791220629b9d0561e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45953 } } peers { permanent_uuid: "d678864a5bbe41fe869538ac35ffeea2" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45385 } }
I20260809 02:15:54.365057 18245 catalog_manager.cc:5495] T 1993a70bf1dd40c3b99039201b62ec5b P 8d22a2768bef4c7d9fc30d0311f732b0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8d22a2768bef4c7d9fc30d0311f732b0 (127.15.75.129). New cstate: current_term: 1 leader_uuid: "8d22a2768bef4c7d9fc30d0311f732b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d22a2768bef4c7d9fc30d0311f732b0" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 36557 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09772a64befd467791220629b9d0561e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d678864a5bbe41fe869538ac35ffeea2" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45385 } health_report { overall_health: UNKNOWN } } }
I20260809 02:15:54.392365 18886 raft_consensus.cc:1261] T 1993a70bf1dd40c3b99039201b62ec5b P d678864a5bbe41fe869538ac35ffeea2 [term 1 FOLLOWER]: Refusing update from remote peer 8d22a2768bef4c7d9fc30d0311f732b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:15:54.392503 18677 raft_consensus.cc:1261] T 1993a70bf1dd40c3b99039201b62ec5b P 09772a64befd467791220629b9d0561e [term 1 FOLLOWER]: Refusing update from remote peer 8d22a2768bef4c7d9fc30d0311f732b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:15:54.392807 18995 consensus_queue.cc:1035] T 1993a70bf1dd40c3b99039201b62ec5b P 8d22a2768bef4c7d9fc30d0311f732b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d678864a5bbe41fe869538ac35ffeea2" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:15:54.392949 19005 consensus_queue.cc:1035] T 1993a70bf1dd40c3b99039201b62ec5b P 8d22a2768bef4c7d9fc30d0311f732b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09772a64befd467791220629b9d0561e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 02:15:54.405375 19035 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 02:15:55.956032 18255 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43022:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:15:55.956180 18255 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43022:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:15:55.958460 18255 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e222f3caa8a145b9ad29749cb134b63c: Sending DeleteTablet for 3 replicas of tablet 1993a70bf1dd40c3b99039201b62ec5b
I20260809 02:15:55.958989 18447 tablet_service.cc:1514] Processing DeleteTablet for tablet 1993a70bf1dd40c3b99039201b62ec5b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:15:55 UTC) from {username='kudu', principal='kudu/127.15.75.190@KRBTEST.COM'} at 127.0.0.1:53630
I20260809 02:15:55.959070 18648 tablet_service.cc:1514] Processing DeleteTablet for tablet 1993a70bf1dd40c3b99039201b62ec5b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:15:55 UTC) from {username='kudu', principal='kudu/127.15.75.190@KRBTEST.COM'} at 127.0.0.1:48474
I20260809 02:15:55.959175 18857 tablet_service.cc:1514] Processing DeleteTablet for tablet 1993a70bf1dd40c3b99039201b62ec5b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:15:55 UTC) from {username='kudu', principal='kudu/127.15.75.190@KRBTEST.COM'} at 127.0.0.1:55542
I20260809 02:15:55.959357 19106 tablet_replica.cc:330] T 1993a70bf1dd40c3b99039201b62ec5b P 8d22a2768bef4c7d9fc30d0311f732b0: stopping tablet replica
I20260809 02:15:55.959503 19106 raft_consensus.cc:2229] T 1993a70bf1dd40c3b99039201b62ec5b P 8d22a2768bef4c7d9fc30d0311f732b0 [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:15:55.959550 19108 tablet_replica.cc:330] T 1993a70bf1dd40c3b99039201b62ec5b P d678864a5bbe41fe869538ac35ffeea2: stopping tablet replica
I20260809 02:15:55.959657 19108 raft_consensus.cc:2229] T 1993a70bf1dd40c3b99039201b62ec5b P d678864a5bbe41fe869538ac35ffeea2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:15:55.959690 19106 raft_consensus.cc:2258] T 1993a70bf1dd40c3b99039201b62ec5b P 8d22a2768bef4c7d9fc30d0311f732b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:15:55.959760 19108 raft_consensus.cc:2258] T 1993a70bf1dd40c3b99039201b62ec5b P d678864a5bbe41fe869538ac35ffeea2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:15:55.960449 19106 ts_tablet_manager.cc:1905] T 1993a70bf1dd40c3b99039201b62ec5b P 8d22a2768bef4c7d9fc30d0311f732b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:15:55.963093 19108 ts_tablet_manager.cc:1905] T 1993a70bf1dd40c3b99039201b62ec5b P d678864a5bbe41fe869538ac35ffeea2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:15:55.963846 19106 ts_tablet_manager.cc:1918] T 1993a70bf1dd40c3b99039201b62ec5b P 8d22a2768bef4c7d9fc30d0311f732b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 02:15:55.963935 19106 log.cc:1196] T 1993a70bf1dd40c3b99039201b62ec5b P 8d22a2768bef4c7d9fc30d0311f732b0: Deleting WAL directory at /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/minicluster-data/ts-0/wal/wals/1993a70bf1dd40c3b99039201b62ec5b
I20260809 02:15:55.964267 19106 ts_tablet_manager.cc:1939] T 1993a70bf1dd40c3b99039201b62ec5b P 8d22a2768bef4c7d9fc30d0311f732b0: Deleting consensus metadata
I20260809 02:15:55.965042 18234 catalog_manager.cc:4841] TS 8d22a2768bef4c7d9fc30d0311f732b0 (127.15.75.129:36557): tablet 1993a70bf1dd40c3b99039201b62ec5b (table test-table [id=147c9a6457ea4a4cbf9bbddbc62a9c6a]) successfully deleted
I20260809 02:15:55.965360 19108 ts_tablet_manager.cc:1918] T 1993a70bf1dd40c3b99039201b62ec5b P d678864a5bbe41fe869538ac35ffeea2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 02:15:55.965452 19108 log.cc:1196] T 1993a70bf1dd40c3b99039201b62ec5b P d678864a5bbe41fe869538ac35ffeea2: Deleting WAL directory at /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/minicluster-data/ts-2/wal/wals/1993a70bf1dd40c3b99039201b62ec5b
I20260809 02:15:55.965767 19108 ts_tablet_manager.cc:1939] T 1993a70bf1dd40c3b99039201b62ec5b P d678864a5bbe41fe869538ac35ffeea2: Deleting consensus metadata
I20260809 02:15:55.966418 19107 tablet_replica.cc:330] T 1993a70bf1dd40c3b99039201b62ec5b P 09772a64befd467791220629b9d0561e: stopping tablet replica
I20260809 02:15:55.966408 18229 catalog_manager.cc:4841] TS d678864a5bbe41fe869538ac35ffeea2 (127.15.75.131:45385): tablet 1993a70bf1dd40c3b99039201b62ec5b (table test-table [id=147c9a6457ea4a4cbf9bbddbc62a9c6a]) successfully deleted
I20260809 02:15:55.966568 19107 raft_consensus.cc:2229] T 1993a70bf1dd40c3b99039201b62ec5b P 09772a64befd467791220629b9d0561e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:15:55.966708 19107 raft_consensus.cc:2258] T 1993a70bf1dd40c3b99039201b62ec5b P 09772a64befd467791220629b9d0561e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:15:55.967300 19107 ts_tablet_manager.cc:1905] T 1993a70bf1dd40c3b99039201b62ec5b P 09772a64befd467791220629b9d0561e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:15:55.969385 19107 ts_tablet_manager.cc:1918] T 1993a70bf1dd40c3b99039201b62ec5b P 09772a64befd467791220629b9d0561e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 02:15:55.969480 19107 log.cc:1196] T 1993a70bf1dd40c3b99039201b62ec5b P 09772a64befd467791220629b9d0561e: Deleting WAL directory at /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/minicluster-data/ts-1/wal/wals/1993a70bf1dd40c3b99039201b62ec5b
I20260809 02:15:55.969709 19107 ts_tablet_manager.cc:1939] T 1993a70bf1dd40c3b99039201b62ec5b P 09772a64befd467791220629b9d0561e: Deleting consensus metadata
I20260809 02:15:55.970337 18236 catalog_manager.cc:4841] TS 09772a64befd467791220629b9d0561e (127.15.75.130:45953): tablet 1993a70bf1dd40c3b99039201b62ec5b (table test-table [id=147c9a6457ea4a4cbf9bbddbc62a9c6a]) successfully deleted
Aug 09 02:15:55 dist-test-slave-s11t krb5kdc[18181](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241755, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 02:15:55 dist-test-slave-s11t krb5kdc[18181](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241755, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:15:55.988437 19120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:55.979794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54402 (local address 127.15.75.190:44355)
0809 02:15:55.980068 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:55.980074 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:55.980090 (+ 16us) server_negotiation.cc:407] Connection header received
0809 02:15:55.980223 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:55.980225 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:55.980268 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:55.980354 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:55.981068 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:55.981803 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:55.982613 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:55.982855 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:55.982935 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:55.985469 (+ 2534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:15:55.985472 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:15:55.985473 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:15:55.985494 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:15:55.986979 (+ 1485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:55.987342 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:55.987344 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:55.987346 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:55.987427 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:55.987654 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:55.987656 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:55.987657 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:55.987814 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:15:55.987913 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:55.988106 (+ 193us) server_negotiation.cc:299] Negotiation successful
0809 02:15:55.988203 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":129,"threads_started":1}
I20260809 02:15:55.990826 18255 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54402:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:15:55.991199 18255 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:15:55.995877 18447 tablet_service.cc:1467] Processing CreateTablet for tablet 84162f756968496a96a42ce84317a6bc (DEFAULT_TABLE table=test-table [id=537ef0e86d744c8688b044ff34406d51]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:15:55.995937 18648 tablet_service.cc:1467] Processing CreateTablet for tablet 84162f756968496a96a42ce84317a6bc (DEFAULT_TABLE table=test-table [id=537ef0e86d744c8688b044ff34406d51]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:15:55.996112 18857 tablet_service.cc:1467] Processing CreateTablet for tablet 84162f756968496a96a42ce84317a6bc (DEFAULT_TABLE table=test-table [id=537ef0e86d744c8688b044ff34406d51]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:15:55.996196 18648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84162f756968496a96a42ce84317a6bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:15:55.996196 18447 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84162f756968496a96a42ce84317a6bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:15:55.996419 18857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84162f756968496a96a42ce84317a6bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:15:55.998654 19121 tablet_bootstrap.cc:492] T 84162f756968496a96a42ce84317a6bc P d678864a5bbe41fe869538ac35ffeea2: Bootstrap starting.
I20260809 02:15:55.998889 19126 tablet_bootstrap.cc:492] T 84162f756968496a96a42ce84317a6bc P 8d22a2768bef4c7d9fc30d0311f732b0: Bootstrap starting.
I20260809 02:15:55.998987 19123 tablet_bootstrap.cc:492] T 84162f756968496a96a42ce84317a6bc P 09772a64befd467791220629b9d0561e: Bootstrap starting.
I20260809 02:15:55.999764 19126 tablet_bootstrap.cc:654] T 84162f756968496a96a42ce84317a6bc P 8d22a2768bef4c7d9fc30d0311f732b0: Neither blocks nor log segments found. Creating new log.
I20260809 02:15:55.999764 19123 tablet_bootstrap.cc:654] T 84162f756968496a96a42ce84317a6bc P 09772a64befd467791220629b9d0561e: Neither blocks nor log segments found. Creating new log.
I20260809 02:15:56.000047 19121 tablet_bootstrap.cc:654] T 84162f756968496a96a42ce84317a6bc P d678864a5bbe41fe869538ac35ffeea2: Neither blocks nor log segments found. Creating new log.
I20260809 02:15:56.000941 19126 tablet_bootstrap.cc:492] T 84162f756968496a96a42ce84317a6bc P 8d22a2768bef4c7d9fc30d0311f732b0: No bootstrap required, opened a new log
I20260809 02:15:56.000941 19123 tablet_bootstrap.cc:492] T 84162f756968496a96a42ce84317a6bc P 09772a64befd467791220629b9d0561e: No bootstrap required, opened a new log
I20260809 02:15:56.001029 19123 ts_tablet_manager.cc:1397] T 84162f756968496a96a42ce84317a6bc P 09772a64befd467791220629b9d0561e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 02:15:56.001029 19126 ts_tablet_manager.cc:1397] T 84162f756968496a96a42ce84317a6bc P 8d22a2768bef4c7d9fc30d0311f732b0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 02:15:56.001313 19121 tablet_bootstrap.cc:492] T 84162f756968496a96a42ce84317a6bc P d678864a5bbe41fe869538ac35ffeea2: No bootstrap required, opened a new log
I20260809 02:15:56.001418 19121 ts_tablet_manager.cc:1397] T 84162f756968496a96a42ce84317a6bc P d678864a5bbe41fe869538ac35ffeea2: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 02:15:56.001418 19126 raft_consensus.cc:348] T 84162f756968496a96a42ce84317a6bc P 8d22a2768bef4c7d9fc30d0311f732b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d22a2768bef4c7d9fc30d0311f732b0" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 36557 } } peers { permanent_uuid: "09772a64befd467791220629b9d0561e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45953 } } peers { permanent_uuid: "d678864a5bbe41fe869538ac35ffeea2" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45385 } }
I20260809 02:15:56.001418 19123 raft_consensus.cc:348] T 84162f756968496a96a42ce84317a6bc P 09772a64befd467791220629b9d0561e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d22a2768bef4c7d9fc30d0311f732b0" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 36557 } } peers { permanent_uuid: "09772a64befd467791220629b9d0561e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45953 } } peers { permanent_uuid: "d678864a5bbe41fe869538ac35ffeea2" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45385 } }
I20260809 02:15:56.001523 19126 raft_consensus.cc:374] T 84162f756968496a96a42ce84317a6bc P 8d22a2768bef4c7d9fc30d0311f732b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:15:56.001523 19123 raft_consensus.cc:374] T 84162f756968496a96a42ce84317a6bc P 09772a64befd467791220629b9d0561e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:15:56.001557 19123 raft_consensus.cc:729] T 84162f756968496a96a42ce84317a6bc P 09772a64befd467791220629b9d0561e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09772a64befd467791220629b9d0561e, State: Initialized, Role: FOLLOWER
I20260809 02:15:56.001557 19126 raft_consensus.cc:729] T 84162f756968496a96a42ce84317a6bc P 8d22a2768bef4c7d9fc30d0311f732b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d22a2768bef4c7d9fc30d0311f732b0, State: Initialized, Role: FOLLOWER
I20260809 02:15:56.001680 19123 consensus_queue.cc:260] T 84162f756968496a96a42ce84317a6bc P 09772a64befd467791220629b9d0561e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d22a2768bef4c7d9fc30d0311f732b0" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 36557 } } peers { permanent_uuid: "09772a64befd467791220629b9d0561e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45953 } } peers { permanent_uuid: "d678864a5bbe41fe869538ac35ffeea2" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45385 } }
I20260809 02:15:56.001705 19126 consensus_queue.cc:260] T 84162f756968496a96a42ce84317a6bc P 8d22a2768bef4c7d9fc30d0311f732b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d22a2768bef4c7d9fc30d0311f732b0" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 36557 } } peers { permanent_uuid: "09772a64befd467791220629b9d0561e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45953 } } peers { permanent_uuid: "d678864a5bbe41fe869538ac35ffeea2" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45385 } }
I20260809 02:15:56.001879 19121 raft_consensus.cc:348] T 84162f756968496a96a42ce84317a6bc P d678864a5bbe41fe869538ac35ffeea2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d22a2768bef4c7d9fc30d0311f732b0" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 36557 } } peers { permanent_uuid: "09772a64befd467791220629b9d0561e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45953 } } peers { permanent_uuid: "d678864a5bbe41fe869538ac35ffeea2" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45385 } }
I20260809 02:15:56.001952 19123 ts_tablet_manager.cc:1428] T 84162f756968496a96a42ce84317a6bc P 09772a64befd467791220629b9d0561e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 02:15:56.001997 19121 raft_consensus.cc:374] T 84162f756968496a96a42ce84317a6bc P d678864a5bbe41fe869538ac35ffeea2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:15:56.002038 19121 raft_consensus.cc:729] T 84162f756968496a96a42ce84317a6bc P d678864a5bbe41fe869538ac35ffeea2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d678864a5bbe41fe869538ac35ffeea2, State: Initialized, Role: FOLLOWER
I20260809 02:15:56.002182 19121 consensus_queue.cc:260] T 84162f756968496a96a42ce84317a6bc P d678864a5bbe41fe869538ac35ffeea2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d22a2768bef4c7d9fc30d0311f732b0" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 36557 } } peers { permanent_uuid: "09772a64befd467791220629b9d0561e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45953 } } peers { permanent_uuid: "d678864a5bbe41fe869538ac35ffeea2" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45385 } }
I20260809 02:15:56.002393 19126 ts_tablet_manager.cc:1428] T 84162f756968496a96a42ce84317a6bc P 8d22a2768bef4c7d9fc30d0311f732b0: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260809 02:15:56.002604 19121 ts_tablet_manager.cc:1428] T 84162f756968496a96a42ce84317a6bc P d678864a5bbe41fe869538ac35ffeea2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 02:15:56.150310 19049 raft_consensus.cc:482] T 84162f756968496a96a42ce84317a6bc P 09772a64befd467791220629b9d0561e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:15:56.150444 19049 raft_consensus.cc:504] T 84162f756968496a96a42ce84317a6bc P 09772a64befd467791220629b9d0561e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d22a2768bef4c7d9fc30d0311f732b0" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 36557 } } peers { permanent_uuid: "09772a64befd467791220629b9d0561e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45953 } } peers { permanent_uuid: "d678864a5bbe41fe869538ac35ffeea2" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45385 } }
I20260809 02:15:56.150835 19049 leader_election.cc:288] T 84162f756968496a96a42ce84317a6bc P 09772a64befd467791220629b9d0561e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8d22a2768bef4c7d9fc30d0311f732b0 (127.15.75.129:36557), d678864a5bbe41fe869538ac35ffeea2 (127.15.75.131:45385)
I20260809 02:15:56.151257 18479 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84162f756968496a96a42ce84317a6bc" candidate_uuid: "09772a64befd467791220629b9d0561e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d22a2768bef4c7d9fc30d0311f732b0" is_pre_election: true
I20260809 02:15:56.151268 18886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84162f756968496a96a42ce84317a6bc" candidate_uuid: "09772a64befd467791220629b9d0561e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d678864a5bbe41fe869538ac35ffeea2" is_pre_election: true
I20260809 02:15:56.151432 18479 raft_consensus.cc:2454] T 84162f756968496a96a42ce84317a6bc P 8d22a2768bef4c7d9fc30d0311f732b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09772a64befd467791220629b9d0561e in term 0.
I20260809 02:15:56.151438 18886 raft_consensus.cc:2454] T 84162f756968496a96a42ce84317a6bc P d678864a5bbe41fe869538ac35ffeea2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09772a64befd467791220629b9d0561e in term 0.
I20260809 02:15:56.151813 18585 leader_election.cc:302] T 84162f756968496a96a42ce84317a6bc P 09772a64befd467791220629b9d0561e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09772a64befd467791220629b9d0561e, 8d22a2768bef4c7d9fc30d0311f732b0; no voters:
I20260809 02:15:56.152025 19049 raft_consensus.cc:2781] T 84162f756968496a96a42ce84317a6bc P 09772a64befd467791220629b9d0561e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:15:56.152088 19049 raft_consensus.cc:482] T 84162f756968496a96a42ce84317a6bc P 09772a64befd467791220629b9d0561e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:15:56.152127 19049 raft_consensus.cc:3037] T 84162f756968496a96a42ce84317a6bc P 09772a64befd467791220629b9d0561e [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:15:56.153218 19049 raft_consensus.cc:504] T 84162f756968496a96a42ce84317a6bc P 09772a64befd467791220629b9d0561e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d22a2768bef4c7d9fc30d0311f732b0" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 36557 } } peers { permanent_uuid: "09772a64befd467791220629b9d0561e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45953 } } peers { permanent_uuid: "d678864a5bbe41fe869538ac35ffeea2" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45385 } }
I20260809 02:15:56.153540 19049 leader_election.cc:288] T 84162f756968496a96a42ce84317a6bc P 09772a64befd467791220629b9d0561e [CANDIDATE]: Term 1 election: Requested vote from peers 8d22a2768bef4c7d9fc30d0311f732b0 (127.15.75.129:36557), d678864a5bbe41fe869538ac35ffeea2 (127.15.75.131:45385)
I20260809 02:15:56.153831 18479 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84162f756968496a96a42ce84317a6bc" candidate_uuid: "09772a64befd467791220629b9d0561e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d22a2768bef4c7d9fc30d0311f732b0"
I20260809 02:15:56.153980 18479 raft_consensus.cc:3037] T 84162f756968496a96a42ce84317a6bc P 8d22a2768bef4c7d9fc30d0311f732b0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:15:56.153980 18886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84162f756968496a96a42ce84317a6bc" candidate_uuid: "09772a64befd467791220629b9d0561e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d678864a5bbe41fe869538ac35ffeea2"
I20260809 02:15:56.154086 18886 raft_consensus.cc:3037] T 84162f756968496a96a42ce84317a6bc P d678864a5bbe41fe869538ac35ffeea2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:15:56.155172 18479 raft_consensus.cc:2454] T 84162f756968496a96a42ce84317a6bc P 8d22a2768bef4c7d9fc30d0311f732b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09772a64befd467791220629b9d0561e in term 1.
I20260809 02:15:56.155172 18886 raft_consensus.cc:2454] T 84162f756968496a96a42ce84317a6bc P d678864a5bbe41fe869538ac35ffeea2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09772a64befd467791220629b9d0561e in term 1.
I20260809 02:15:56.155450 18583 leader_election.cc:302] T 84162f756968496a96a42ce84317a6bc P 09772a64befd467791220629b9d0561e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09772a64befd467791220629b9d0561e, d678864a5bbe41fe869538ac35ffeea2; no voters:
I20260809 02:15:56.155745 19049 raft_consensus.cc:2781] T 84162f756968496a96a42ce84317a6bc P 09772a64befd467791220629b9d0561e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:15:56.155849 19049 raft_consensus.cc:686] T 84162f756968496a96a42ce84317a6bc P 09772a64befd467791220629b9d0561e [term 1 LEADER]: Becoming Leader. State: Replica: 09772a64befd467791220629b9d0561e, State: Running, Role: LEADER
I20260809 02:15:56.155997 19049 consensus_queue.cc:237] T 84162f756968496a96a42ce84317a6bc P 09772a64befd467791220629b9d0561e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d22a2768bef4c7d9fc30d0311f732b0" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 36557 } } peers { permanent_uuid: "09772a64befd467791220629b9d0561e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45953 } } peers { permanent_uuid: "d678864a5bbe41fe869538ac35ffeea2" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45385 } }
I20260809 02:15:56.157600 18245 catalog_manager.cc:5495] T 84162f756968496a96a42ce84317a6bc P 09772a64befd467791220629b9d0561e reported cstate change: term changed from 0 to 1, leader changed from <none> to 09772a64befd467791220629b9d0561e (127.15.75.130). New cstate: current_term: 1 leader_uuid: "09772a64befd467791220629b9d0561e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d22a2768bef4c7d9fc30d0311f732b0" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 36557 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09772a64befd467791220629b9d0561e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d678864a5bbe41fe869538ac35ffeea2" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45385 } health_report { overall_health: UNKNOWN } } }
I20260809 02:15:56.197458 19134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:56.194217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34080 (local address 127.15.75.130:45953)
0809 02:15:56.194495 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:56.194502 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:56.194513 (+ 11us) server_negotiation.cc:407] Connection header received
0809 02:15:56.194551 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:56.194553 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:56.194593 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:56.194673 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:15:56.194929 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:56.195448 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:56.196352 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:56.196584 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:56.196671 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:56.197014 (+ 343us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:15:56.197083 (+ 69us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:15:56.197141 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:56.197228 (+ 87us) server_negotiation.cc:299] Negotiation successful
0809 02:15:56.197274 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":140,"threads_started":1}
I20260809 02:15:56.205849 19134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:56.203101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.130:59025 (local address 127.15.75.130:45953)
0809 02:15:56.203229 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:56.203232 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:56.203287 (+ 55us) server_negotiation.cc:407] Connection header received
0809 02:15:56.203386 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:56.203388 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:56.203425 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:56.203493 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:15:56.203686 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:56.204148 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:56.204928 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:56.205133 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:56.205223 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:56.205447 (+ 224us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:15:56.205505 (+ 58us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:15:56.205559 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:56.205698 (+ 139us) server_negotiation.cc:299] Negotiation successful
0809 02:15:56.205748 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":46}
I20260809 02:15:56.206084 19138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:56.202972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.130:45953 (local address 127.15.75.130:59025)
0809 02:15:56.203211 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:56.203223 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:56.203306 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:56.203514 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:56.203516 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:56.203522 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 02:15:56.203599 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:56.203603 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:56.204252 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:56.204254 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:56.204834 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:56.204841 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:56.205229 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:56.205231 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:56.205328 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:56.205351 (+ 23us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 02:15:56.205607 (+ 256us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 02:15:56.205615 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 02:15:56.205687 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 02:15:56.205890 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"mutex_wait_us":29,"thread_start_us":99,"threads_started":1}
I20260809 02:15:56.211483 18479 raft_consensus.cc:1261] T 84162f756968496a96a42ce84317a6bc P 8d22a2768bef4c7d9fc30d0311f732b0 [term 1 FOLLOWER]: Refusing update from remote peer 09772a64befd467791220629b9d0561e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:15:56.211489 18886 raft_consensus.cc:1261] T 84162f756968496a96a42ce84317a6bc P d678864a5bbe41fe869538ac35ffeea2 [term 1 FOLLOWER]: Refusing update from remote peer 09772a64befd467791220629b9d0561e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:15:56.211911 19084 consensus_queue.cc:1035] T 84162f756968496a96a42ce84317a6bc P 09772a64befd467791220629b9d0561e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d678864a5bbe41fe869538ac35ffeea2" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:15:56.212057 19086 consensus_queue.cc:1035] T 84162f756968496a96a42ce84317a6bc P 09772a64befd467791220629b9d0561e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d22a2768bef4c7d9fc30d0311f732b0" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 36557 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:15:56.246049 18249 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54402:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:15:56.246249 18249 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54402:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:15:56.248602 18249 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e222f3caa8a145b9ad29749cb134b63c: Sending DeleteTablet for 3 replicas of tablet 84162f756968496a96a42ce84317a6bc
I20260809 02:15:56.249116 18447 tablet_service.cc:1514] Processing DeleteTablet for tablet 84162f756968496a96a42ce84317a6bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:15:56 UTC) from {username='kudu', principal='kudu/127.15.75.190@KRBTEST.COM'} at 127.0.0.1:53630
I20260809 02:15:56.249316 19106 tablet_replica.cc:330] T 84162f756968496a96a42ce84317a6bc P 8d22a2768bef4c7d9fc30d0311f732b0: stopping tablet replica
I20260809 02:15:56.249332 18857 tablet_service.cc:1514] Processing DeleteTablet for tablet 84162f756968496a96a42ce84317a6bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:15:56 UTC) from {username='kudu', principal='kudu/127.15.75.190@KRBTEST.COM'} at 127.0.0.1:55542
I20260809 02:15:56.249439 19106 raft_consensus.cc:2229] T 84162f756968496a96a42ce84317a6bc P 8d22a2768bef4c7d9fc30d0311f732b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:15:56.249482 19108 tablet_replica.cc:330] T 84162f756968496a96a42ce84317a6bc P d678864a5bbe41fe869538ac35ffeea2: stopping tablet replica
I20260809 02:15:56.249542 19106 raft_consensus.cc:2258] T 84162f756968496a96a42ce84317a6bc P 8d22a2768bef4c7d9fc30d0311f732b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:15:56.249577 19108 raft_consensus.cc:2229] T 84162f756968496a96a42ce84317a6bc P d678864a5bbe41fe869538ac35ffeea2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:15:56.249672 19108 raft_consensus.cc:2258] T 84162f756968496a96a42ce84317a6bc P d678864a5bbe41fe869538ac35ffeea2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:15:56.250401 19108 ts_tablet_manager.cc:1905] T 84162f756968496a96a42ce84317a6bc P d678864a5bbe41fe869538ac35ffeea2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:15:56.250516 19106 ts_tablet_manager.cc:1905] T 84162f756968496a96a42ce84317a6bc P 8d22a2768bef4c7d9fc30d0311f732b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:15:56.251780 18645 tablet_service.cc:1514] Processing DeleteTablet for tablet 84162f756968496a96a42ce84317a6bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:15:56 UTC) from {username='kudu', principal='kudu/127.15.75.190@KRBTEST.COM'} at 127.0.0.1:48474
I20260809 02:15:56.252820 19107 tablet_replica.cc:330] T 84162f756968496a96a42ce84317a6bc P 09772a64befd467791220629b9d0561e: stopping tablet replica
I20260809 02:15:56.252928 19108 ts_tablet_manager.cc:1918] T 84162f756968496a96a42ce84317a6bc P d678864a5bbe41fe869538ac35ffeea2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 02:15:56.252947 19107 raft_consensus.cc:2229] T 84162f756968496a96a42ce84317a6bc P 09772a64befd467791220629b9d0561e [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:15:56.253015 19108 log.cc:1196] T 84162f756968496a96a42ce84317a6bc P d678864a5bbe41fe869538ac35ffeea2: Deleting WAL directory at /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/minicluster-data/ts-2/wal/wals/84162f756968496a96a42ce84317a6bc
I20260809 02:15:56.253217 19107 raft_consensus.cc:2258] T 84162f756968496a96a42ce84317a6bc P 09772a64befd467791220629b9d0561e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:15:56.253823 19106 ts_tablet_manager.cc:1918] T 84162f756968496a96a42ce84317a6bc P 8d22a2768bef4c7d9fc30d0311f732b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 02:15:56.253906 19106 log.cc:1196] T 84162f756968496a96a42ce84317a6bc P 8d22a2768bef4c7d9fc30d0311f732b0: Deleting WAL directory at /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/minicluster-data/ts-0/wal/wals/84162f756968496a96a42ce84317a6bc
I20260809 02:15:56.253999 19107 ts_tablet_manager.cc:1905] T 84162f756968496a96a42ce84317a6bc P 09772a64befd467791220629b9d0561e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:15:56.254159 19106 ts_tablet_manager.cc:1939] T 84162f756968496a96a42ce84317a6bc P 8d22a2768bef4c7d9fc30d0311f732b0: Deleting consensus metadata
I20260809 02:15:56.254765 18234 catalog_manager.cc:4841] TS 8d22a2768bef4c7d9fc30d0311f732b0 (127.15.75.129:36557): tablet 84162f756968496a96a42ce84317a6bc (table test-table [id=537ef0e86d744c8688b044ff34406d51]) successfully deleted
I20260809 02:15:56.256191 19107 ts_tablet_manager.cc:1918] T 84162f756968496a96a42ce84317a6bc P 09772a64befd467791220629b9d0561e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 02:15:56.256291 19107 log.cc:1196] T 84162f756968496a96a42ce84317a6bc P 09772a64befd467791220629b9d0561e: Deleting WAL directory at /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/minicluster-data/ts-1/wal/wals/84162f756968496a96a42ce84317a6bc
I20260809 02:15:56.256542 19107 ts_tablet_manager.cc:1939] T 84162f756968496a96a42ce84317a6bc P 09772a64befd467791220629b9d0561e: Deleting consensus metadata
I20260809 02:15:56.253289 19108 ts_tablet_manager.cc:1939] T 84162f756968496a96a42ce84317a6bc P d678864a5bbe41fe869538ac35ffeea2: Deleting consensus metadata
I20260809 02:15:56.257169 18236 catalog_manager.cc:4841] TS 09772a64befd467791220629b9d0561e (127.15.75.130:45953): tablet 84162f756968496a96a42ce84317a6bc (table test-table [id=537ef0e86d744c8688b044ff34406d51]) successfully deleted
I20260809 02:15:56.257313 18249 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54402:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:15:56.257691 18249 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:15:56.257737 18229 catalog_manager.cc:4841] TS d678864a5bbe41fe869538ac35ffeea2 (127.15.75.131:45385): tablet 84162f756968496a96a42ce84317a6bc (table test-table [id=537ef0e86d744c8688b044ff34406d51]) successfully deleted
I20260809 02:15:56.262106 18447 tablet_service.cc:1467] Processing CreateTablet for tablet fb7b616d7b2b4a1d8d73fb1aa073dbf0 (DEFAULT_TABLE table=test-table [id=08f9310c756c43afa1ed750626eb0a3d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:15:56.262355 18447 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb7b616d7b2b4a1d8d73fb1aa073dbf0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:15:56.262326 18857 tablet_service.cc:1467] Processing CreateTablet for tablet fb7b616d7b2b4a1d8d73fb1aa073dbf0 (DEFAULT_TABLE table=test-table [id=08f9310c756c43afa1ed750626eb0a3d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:15:56.262578 18857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb7b616d7b2b4a1d8d73fb1aa073dbf0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:15:56.264284 19121 tablet_bootstrap.cc:492] T fb7b616d7b2b4a1d8d73fb1aa073dbf0 P d678864a5bbe41fe869538ac35ffeea2: Bootstrap starting.
I20260809 02:15:56.264505 19126 tablet_bootstrap.cc:492] T fb7b616d7b2b4a1d8d73fb1aa073dbf0 P 8d22a2768bef4c7d9fc30d0311f732b0: Bootstrap starting.
I20260809 02:15:56.265214 19121 tablet_bootstrap.cc:654] T fb7b616d7b2b4a1d8d73fb1aa073dbf0 P d678864a5bbe41fe869538ac35ffeea2: Neither blocks nor log segments found. Creating new log.
I20260809 02:15:56.265360 19126 tablet_bootstrap.cc:654] T fb7b616d7b2b4a1d8d73fb1aa073dbf0 P 8d22a2768bef4c7d9fc30d0311f732b0: Neither blocks nor log segments found. Creating new log.
I20260809 02:15:56.266213 18645 tablet_service.cc:1467] Processing CreateTablet for tablet fb7b616d7b2b4a1d8d73fb1aa073dbf0 (DEFAULT_TABLE table=test-table [id=08f9310c756c43afa1ed750626eb0a3d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:15:56.266464 18645 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb7b616d7b2b4a1d8d73fb1aa073dbf0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:15:56.266567 19121 tablet_bootstrap.cc:492] T fb7b616d7b2b4a1d8d73fb1aa073dbf0 P d678864a5bbe41fe869538ac35ffeea2: No bootstrap required, opened a new log
I20260809 02:15:56.266656 19121 ts_tablet_manager.cc:1397] T fb7b616d7b2b4a1d8d73fb1aa073dbf0 P d678864a5bbe41fe869538ac35ffeea2: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 02:15:56.267022 19121 raft_consensus.cc:348] T fb7b616d7b2b4a1d8d73fb1aa073dbf0 P d678864a5bbe41fe869538ac35ffeea2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d22a2768bef4c7d9fc30d0311f732b0" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 36557 } } peers { permanent_uuid: "09772a64befd467791220629b9d0561e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45953 } } peers { permanent_uuid: "d678864a5bbe41fe869538ac35ffeea2" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45385 } }
I20260809 02:15:56.267138 19121 raft_consensus.cc:374] T fb7b616d7b2b4a1d8d73fb1aa073dbf0 P d678864a5bbe41fe869538ac35ffeea2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:15:56.267179 19121 raft_consensus.cc:729] T fb7b616d7b2b4a1d8d73fb1aa073dbf0 P d678864a5bbe41fe869538ac35ffeea2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d678864a5bbe41fe869538ac35ffeea2, State: Initialized, Role: FOLLOWER
I20260809 02:15:56.267323 19121 consensus_queue.cc:260] T fb7b616d7b2b4a1d8d73fb1aa073dbf0 P d678864a5bbe41fe869538ac35ffeea2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d22a2768bef4c7d9fc30d0311f732b0" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 36557 } } peers { permanent_uuid: "09772a64befd467791220629b9d0561e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45953 } } peers { permanent_uuid: "d678864a5bbe41fe869538ac35ffeea2" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45385 } }
I20260809 02:15:56.267588 19121 ts_tablet_manager.cc:1428] T fb7b616d7b2b4a1d8d73fb1aa073dbf0 P d678864a5bbe41fe869538ac35ffeea2: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260809 02:15:56.268637 19123 tablet_bootstrap.cc:492] T fb7b616d7b2b4a1d8d73fb1aa073dbf0 P 09772a64befd467791220629b9d0561e: Bootstrap starting.
I20260809 02:15:56.269447 19123 tablet_bootstrap.cc:654] T fb7b616d7b2b4a1d8d73fb1aa073dbf0 P 09772a64befd467791220629b9d0561e: Neither blocks nor log segments found. Creating new log.
I20260809 02:15:56.270576 19123 tablet_bootstrap.cc:492] T fb7b616d7b2b4a1d8d73fb1aa073dbf0 P 09772a64befd467791220629b9d0561e: No bootstrap required, opened a new log
I20260809 02:15:56.270663 19123 ts_tablet_manager.cc:1397] T fb7b616d7b2b4a1d8d73fb1aa073dbf0 P 09772a64befd467791220629b9d0561e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 02:15:56.271039 19123 raft_consensus.cc:348] T fb7b616d7b2b4a1d8d73fb1aa073dbf0 P 09772a64befd467791220629b9d0561e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d22a2768bef4c7d9fc30d0311f732b0" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 36557 } } peers { permanent_uuid: "09772a64befd467791220629b9d0561e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45953 } } peers { permanent_uuid: "d678864a5bbe41fe869538ac35ffeea2" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45385 } }
I20260809 02:15:56.271148 19123 raft_consensus.cc:374] T fb7b616d7b2b4a1d8d73fb1aa073dbf0 P 09772a64befd467791220629b9d0561e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:15:56.271178 19123 raft_consensus.cc:729] T fb7b616d7b2b4a1d8d73fb1aa073dbf0 P 09772a64befd467791220629b9d0561e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09772a64befd467791220629b9d0561e, State: Initialized, Role: FOLLOWER
I20260809 02:15:56.271294 19123 consensus_queue.cc:260] T fb7b616d7b2b4a1d8d73fb1aa073dbf0 P 09772a64befd467791220629b9d0561e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d22a2768bef4c7d9fc30d0311f732b0" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 36557 } } peers { permanent_uuid: "09772a64befd467791220629b9d0561e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45953 } } peers { permanent_uuid: "d678864a5bbe41fe869538ac35ffeea2" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45385 } }
I20260809 02:15:56.271539 19123 ts_tablet_manager.cc:1428] T fb7b616d7b2b4a1d8d73fb1aa073dbf0 P 09772a64befd467791220629b9d0561e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 02:15:56.272308 19126 tablet_bootstrap.cc:492] T fb7b616d7b2b4a1d8d73fb1aa073dbf0 P 8d22a2768bef4c7d9fc30d0311f732b0: No bootstrap required, opened a new log
I20260809 02:15:56.272397 19126 ts_tablet_manager.cc:1397] T fb7b616d7b2b4a1d8d73fb1aa073dbf0 P 8d22a2768bef4c7d9fc30d0311f732b0: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.001s
I20260809 02:15:56.272797 19126 raft_consensus.cc:348] T fb7b616d7b2b4a1d8d73fb1aa073dbf0 P 8d22a2768bef4c7d9fc30d0311f732b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d22a2768bef4c7d9fc30d0311f732b0" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 36557 } } peers { permanent_uuid: "09772a64befd467791220629b9d0561e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45953 } } peers { permanent_uuid: "d678864a5bbe41fe869538ac35ffeea2" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45385 } }
I20260809 02:15:56.272912 19126 raft_consensus.cc:374] T fb7b616d7b2b4a1d8d73fb1aa073dbf0 P 8d22a2768bef4c7d9fc30d0311f732b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:15:56.272948 19126 raft_consensus.cc:729] T fb7b616d7b2b4a1d8d73fb1aa073dbf0 P 8d22a2768bef4c7d9fc30d0311f732b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d22a2768bef4c7d9fc30d0311f732b0, State: Initialized, Role: FOLLOWER
I20260809 02:15:56.273068 19126 consensus_queue.cc:260] T fb7b616d7b2b4a1d8d73fb1aa073dbf0 P 8d22a2768bef4c7d9fc30d0311f732b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d22a2768bef4c7d9fc30d0311f732b0" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 36557 } } peers { permanent_uuid: "09772a64befd467791220629b9d0561e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45953 } } peers { permanent_uuid: "d678864a5bbe41fe869538ac35ffeea2" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45385 } }
I20260809 02:15:56.273313 19126 ts_tablet_manager.cc:1428] T fb7b616d7b2b4a1d8d73fb1aa073dbf0 P 8d22a2768bef4c7d9fc30d0311f732b0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 02:15:56.295588 19086 raft_consensus.cc:482] T fb7b616d7b2b4a1d8d73fb1aa073dbf0 P 09772a64befd467791220629b9d0561e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:15:56.295706 19086 raft_consensus.cc:504] T fb7b616d7b2b4a1d8d73fb1aa073dbf0 P 09772a64befd467791220629b9d0561e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d22a2768bef4c7d9fc30d0311f732b0" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 36557 } } peers { permanent_uuid: "09772a64befd467791220629b9d0561e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45953 } } peers { permanent_uuid: "d678864a5bbe41fe869538ac35ffeea2" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45385 } }
I20260809 02:15:56.296103 19086 leader_election.cc:288] T fb7b616d7b2b4a1d8d73fb1aa073dbf0 P 09772a64befd467791220629b9d0561e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8d22a2768bef4c7d9fc30d0311f732b0 (127.15.75.129:36557), d678864a5bbe41fe869538ac35ffeea2 (127.15.75.131:45385)
I20260809 02:15:56.296475 18479 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb7b616d7b2b4a1d8d73fb1aa073dbf0" candidate_uuid: "09772a64befd467791220629b9d0561e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d22a2768bef4c7d9fc30d0311f732b0" is_pre_election: true
I20260809 02:15:56.296474 18886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb7b616d7b2b4a1d8d73fb1aa073dbf0" candidate_uuid: "09772a64befd467791220629b9d0561e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d678864a5bbe41fe869538ac35ffeea2" is_pre_election: true
I20260809 02:15:56.296622 18886 raft_consensus.cc:2454] T fb7b616d7b2b4a1d8d73fb1aa073dbf0 P d678864a5bbe41fe869538ac35ffeea2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09772a64befd467791220629b9d0561e in term 0.
I20260809 02:15:56.296627 18479 raft_consensus.cc:2454] T fb7b616d7b2b4a1d8d73fb1aa073dbf0 P 8d22a2768bef4c7d9fc30d0311f732b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09772a64befd467791220629b9d0561e in term 0.
I20260809 02:15:56.296944 18585 leader_election.cc:302] T fb7b616d7b2b4a1d8d73fb1aa073dbf0 P 09772a64befd467791220629b9d0561e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09772a64befd467791220629b9d0561e, 8d22a2768bef4c7d9fc30d0311f732b0; no voters:
I20260809 02:15:56.297137 19086 raft_consensus.cc:2781] T fb7b616d7b2b4a1d8d73fb1aa073dbf0 P 09772a64befd467791220629b9d0561e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:15:56.297219 19086 raft_consensus.cc:482] T fb7b616d7b2b4a1d8d73fb1aa073dbf0 P 09772a64befd467791220629b9d0561e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:15:56.297273 19086 raft_consensus.cc:3037] T fb7b616d7b2b4a1d8d73fb1aa073dbf0 P 09772a64befd467791220629b9d0561e [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:15:56.298310 19086 raft_consensus.cc:504] T fb7b616d7b2b4a1d8d73fb1aa073dbf0 P 09772a64befd467791220629b9d0561e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d22a2768bef4c7d9fc30d0311f732b0" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 36557 } } peers { permanent_uuid: "09772a64befd467791220629b9d0561e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45953 } } peers { permanent_uuid: "d678864a5bbe41fe869538ac35ffeea2" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45385 } }
I20260809 02:15:56.298653 19086 leader_election.cc:288] T fb7b616d7b2b4a1d8d73fb1aa073dbf0 P 09772a64befd467791220629b9d0561e [CANDIDATE]: Term 1 election: Requested vote from peers 8d22a2768bef4c7d9fc30d0311f732b0 (127.15.75.129:36557), d678864a5bbe41fe869538ac35ffeea2 (127.15.75.131:45385)
I20260809 02:15:56.298975 18479 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb7b616d7b2b4a1d8d73fb1aa073dbf0" candidate_uuid: "09772a64befd467791220629b9d0561e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d22a2768bef4c7d9fc30d0311f732b0"
I20260809 02:15:56.298969 18886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb7b616d7b2b4a1d8d73fb1aa073dbf0" candidate_uuid: "09772a64befd467791220629b9d0561e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d678864a5bbe41fe869538ac35ffeea2"
I20260809 02:15:56.299084 18479 raft_consensus.cc:3037] T fb7b616d7b2b4a1d8d73fb1aa073dbf0 P 8d22a2768bef4c7d9fc30d0311f732b0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:15:56.299084 18886 raft_consensus.cc:3037] T fb7b616d7b2b4a1d8d73fb1aa073dbf0 P d678864a5bbe41fe869538ac35ffeea2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:15:56.300184 18886 raft_consensus.cc:2454] T fb7b616d7b2b4a1d8d73fb1aa073dbf0 P d678864a5bbe41fe869538ac35ffeea2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09772a64befd467791220629b9d0561e in term 1.
I20260809 02:15:56.300184 18479 raft_consensus.cc:2454] T fb7b616d7b2b4a1d8d73fb1aa073dbf0 P 8d22a2768bef4c7d9fc30d0311f732b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09772a64befd467791220629b9d0561e in term 1.
I20260809 02:15:56.300479 18585 leader_election.cc:302] T fb7b616d7b2b4a1d8d73fb1aa073dbf0 P 09772a64befd467791220629b9d0561e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 09772a64befd467791220629b9d0561e, 8d22a2768bef4c7d9fc30d0311f732b0, d678864a5bbe41fe869538ac35ffeea2; no voters:
I20260809 02:15:56.300624 19086 raft_consensus.cc:2781] T fb7b616d7b2b4a1d8d73fb1aa073dbf0 P 09772a64befd467791220629b9d0561e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:15:56.300738 19086 raft_consensus.cc:686] T fb7b616d7b2b4a1d8d73fb1aa073dbf0 P 09772a64befd467791220629b9d0561e [term 1 LEADER]: Becoming Leader. State: Replica: 09772a64befd467791220629b9d0561e, State: Running, Role: LEADER
I20260809 02:15:56.300870 19086 consensus_queue.cc:237] T fb7b616d7b2b4a1d8d73fb1aa073dbf0 P 09772a64befd467791220629b9d0561e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d22a2768bef4c7d9fc30d0311f732b0" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 36557 } } peers { permanent_uuid: "09772a64befd467791220629b9d0561e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45953 } } peers { permanent_uuid: "d678864a5bbe41fe869538ac35ffeea2" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45385 } }
I20260809 02:15:56.302620 18249 catalog_manager.cc:5495] T fb7b616d7b2b4a1d8d73fb1aa073dbf0 P 09772a64befd467791220629b9d0561e reported cstate change: term changed from 0 to 1, leader changed from <none> to 09772a64befd467791220629b9d0561e (127.15.75.130). New cstate: current_term: 1 leader_uuid: "09772a64befd467791220629b9d0561e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d22a2768bef4c7d9fc30d0311f732b0" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 36557 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09772a64befd467791220629b9d0561e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d678864a5bbe41fe869538ac35ffeea2" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45385 } health_report { overall_health: UNKNOWN } } }
I20260809 02:15:56.326723 18479 raft_consensus.cc:1261] T fb7b616d7b2b4a1d8d73fb1aa073dbf0 P 8d22a2768bef4c7d9fc30d0311f732b0 [term 1 FOLLOWER]: Refusing update from remote peer 09772a64befd467791220629b9d0561e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:15:56.327567 19086 consensus_queue.cc:1035] T fb7b616d7b2b4a1d8d73fb1aa073dbf0 P 09772a64befd467791220629b9d0561e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d22a2768bef4c7d9fc30d0311f732b0" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 36557 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:15:56.328054 18886 raft_consensus.cc:1261] T fb7b616d7b2b4a1d8d73fb1aa073dbf0 P d678864a5bbe41fe869538ac35ffeea2 [term 1 FOLLOWER]: Refusing update from remote peer 09772a64befd467791220629b9d0561e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:15:56.328423 19049 consensus_queue.cc:1035] T fb7b616d7b2b4a1d8d73fb1aa073dbf0 P 09772a64befd467791220629b9d0561e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d678864a5bbe41fe869538ac35ffeea2" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 02:15:56.341559 19076 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 02:15:57.893445 18245 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54402:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:15:57.893644 18245 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54402:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:15:57.896404 18245 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e222f3caa8a145b9ad29749cb134b63c: Sending DeleteTablet for 3 replicas of tablet fb7b616d7b2b4a1d8d73fb1aa073dbf0
I20260809 02:15:57.897055 18648 tablet_service.cc:1514] Processing DeleteTablet for tablet fb7b616d7b2b4a1d8d73fb1aa073dbf0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:15:57 UTC) from {username='kudu', principal='kudu/127.15.75.190@KRBTEST.COM'} at 127.0.0.1:48474
I20260809 02:15:57.897055 18447 tablet_service.cc:1514] Processing DeleteTablet for tablet fb7b616d7b2b4a1d8d73fb1aa073dbf0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:15:57 UTC) from {username='kudu', principal='kudu/127.15.75.190@KRBTEST.COM'} at 127.0.0.1:53630
I20260809 02:15:57.897197 18857 tablet_service.cc:1514] Processing DeleteTablet for tablet fb7b616d7b2b4a1d8d73fb1aa073dbf0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:15:57 UTC) from {username='kudu', principal='kudu/127.15.75.190@KRBTEST.COM'} at 127.0.0.1:55542
I20260809 02:15:57.897380 19179 tablet_replica.cc:330] T fb7b616d7b2b4a1d8d73fb1aa073dbf0 P 09772a64befd467791220629b9d0561e: stopping tablet replica
I20260809 02:15:57.897505 19181 tablet_replica.cc:330] T fb7b616d7b2b4a1d8d73fb1aa073dbf0 P d678864a5bbe41fe869538ac35ffeea2: stopping tablet replica
I20260809 02:15:57.897519 19179 raft_consensus.cc:2229] T fb7b616d7b2b4a1d8d73fb1aa073dbf0 P 09772a64befd467791220629b9d0561e [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:15:57.897535 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 18349
I20260809 02:15:57.897619 19181 raft_consensus.cc:2229] T fb7b616d7b2b4a1d8d73fb1aa073dbf0 P d678864a5bbe41fe869538ac35ffeea2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:15:57.905236 19181 raft_consensus.cc:2258] T fb7b616d7b2b4a1d8d73fb1aa073dbf0 P d678864a5bbe41fe869538ac35ffeea2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:15:57.906234 19181 ts_tablet_manager.cc:1905] T fb7b616d7b2b4a1d8d73fb1aa073dbf0 P d678864a5bbe41fe869538ac35ffeea2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:15:57.909214 19181 ts_tablet_manager.cc:1918] T fb7b616d7b2b4a1d8d73fb1aa073dbf0 P d678864a5bbe41fe869538ac35ffeea2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 02:15:57.909293 19181 log.cc:1196] T fb7b616d7b2b4a1d8d73fb1aa073dbf0 P d678864a5bbe41fe869538ac35ffeea2: Deleting WAL directory at /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/minicluster-data/ts-2/wal/wals/fb7b616d7b2b4a1d8d73fb1aa073dbf0
I20260809 02:15:57.909600 19181 ts_tablet_manager.cc:1939] T fb7b616d7b2b4a1d8d73fb1aa073dbf0 P d678864a5bbe41fe869538ac35ffeea2: Deleting consensus metadata
I20260809 02:15:57.918107 18229 catalog_manager.cc:4841] TS d678864a5bbe41fe869538ac35ffeea2 (127.15.75.131:45385): tablet fb7b616d7b2b4a1d8d73fb1aa073dbf0 (table test-table [id=08f9310c756c43afa1ed750626eb0a3d]) successfully deleted
I20260809 02:15:57.918447 19179 raft_consensus.cc:2258] T fb7b616d7b2b4a1d8d73fb1aa073dbf0 P 09772a64befd467791220629b9d0561e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:15:57.919706 19179 ts_tablet_manager.cc:1905] T fb7b616d7b2b4a1d8d73fb1aa073dbf0 P 09772a64befd467791220629b9d0561e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:15:57.921952 19179 ts_tablet_manager.cc:1918] T fb7b616d7b2b4a1d8d73fb1aa073dbf0 P 09772a64befd467791220629b9d0561e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 02:15:57.922052 19179 log.cc:1196] T fb7b616d7b2b4a1d8d73fb1aa073dbf0 P 09772a64befd467791220629b9d0561e: Deleting WAL directory at /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241741788589-15662-0/minicluster-data/ts-1/wal/wals/fb7b616d7b2b4a1d8d73fb1aa073dbf0
I20260809 02:15:57.922330 19179 ts_tablet_manager.cc:1939] T fb7b616d7b2b4a1d8d73fb1aa073dbf0 P 09772a64befd467791220629b9d0561e: Deleting consensus metadata
I20260809 02:15:57.923079 18236 catalog_manager.cc:4841] TS 09772a64befd467791220629b9d0561e (127.15.75.130:45953): tablet fb7b616d7b2b4a1d8d73fb1aa073dbf0 (table test-table [id=08f9310c756c43afa1ed750626eb0a3d]) successfully deleted
W20260809 02:15:57.928493 18234 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.75.129:36557 (error 108)
W20260809 02:15:57.929785 19183 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:15:57.929289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.129:36557 (local address 127.0.0.1:45894)
0809 02:15:57.929544 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:57.929642 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.129:36557: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":95,"threads_started":1}
I20260809 02:15:57.929952 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 18559
W20260809 02:15:57.937168 18234 catalog_manager.cc:4584] TS 8d22a2768bef4c7d9fc30d0311f732b0 (127.15.75.129:36557): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fb7b616d7b2b4a1d8d73fb1aa073dbf0: Network error: Client connection negotiation failed: client connection to 127.15.75.129:36557: connect: Connection reset by peer (error 104)
I20260809 02:15:57.951488 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 18763
I20260809 02:15:57.966781 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 18196
2026-08-09T02:15:57Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6463 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:15:58 dist-test-slave-s11t krb5kdc[19186](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:15:58 dist-test-slave-s11t krb5kdc[19186](info): set up 2 sockets
Aug 09 02:15:58 dist-test-slave-s11t krb5kdc[19186](info): commencing operation
krb5kdc: starting...
W20260809 02:15:59.222981 15662 mini_kdc.cc:121] Time spent starting KDC: real 1.224s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Aug 09 02:15:59 dist-test-slave-s11t krb5kdc[19186](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241759, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:15:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:15:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.75.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab.
Entry for principal kudu/127.15.75.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab.
I20260809 02:15:59.342816 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:34113
--webserver_interface=127.15.75.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab
--principal=kudu/127.15.75.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:37085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.75.190:34113
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:15:59.451321 19209 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:15:59.451558 19209 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:15:59.451597 19209 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:15:59.455416 19209 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:15:59.455466 19209 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:15:59.455484 19209 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:15:59.455500 19209 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:15:59.455515 19209 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:15:59.460644 19209 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:37085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.75.190:34113
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab
--principal=kudu/127.15.75.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:34113
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.75.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19209
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:15:59.461585 19209 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:15:59.462479 19209 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:15:59.470916 19219 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:15:59.470924 19216 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:15:59.470988 19217 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:15:59.471527 19209 server_base.cc:1034] running on GCE node
I20260809 02:15:59.472074 19209 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:15:59.473196 19209 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:15:59.474380 19209 hybrid_clock.cc:648] HybridClock initialized: now 1786241759474351 us; error 43 us; skew 500 ppm
Aug 09 02:15:59 dist-test-slave-s11t krb5kdc[19186](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241759, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:15:59.477015 19209 init.cc:377] Logged in from keytab as kudu/127.15.75.190@KRBTEST.COM (short username kudu)
I20260809 02:15:59.477931 19209 webserver.cc:460] Webserver started at http://127.15.75.190:33913/ using document root <none> and password file <none>
I20260809 02:15:59.478461 19209 fs_manager.cc:359] Metadata directory not provided
I20260809 02:15:59.478547 19209 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:15:59.478775 19209 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:15:59.480799 19209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/minicluster-data/master-0/data/instance:
uuid: "512f28f7969f47b195c73785aad771ab"
format_stamp: "Formatted at 2026-08-09 02:15:59 on dist-test-slave-s11t"
server_key: "c702da4bc8467c950649298fe470606a"
server_key_iv: "2898f9412f331c3c1c60140d24167f42"
server_key_version: "encryptionkey@0"
I20260809 02:15:59.481379 19209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/minicluster-data/master-0/wal/instance:
uuid: "512f28f7969f47b195c73785aad771ab"
format_stamp: "Formatted at 2026-08-09 02:15:59 on dist-test-slave-s11t"
server_key: "c702da4bc8467c950649298fe470606a"
server_key_iv: "2898f9412f331c3c1c60140d24167f42"
server_key_version: "encryptionkey@0"
I20260809 02:15:59.484869 19209 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260809 02:15:59.487125 19229 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:59.488300 19209 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 02:15:59.488433 19209 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/minicluster-data/master-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/minicluster-data/master-0/wal
uuid: "512f28f7969f47b195c73785aad771ab"
format_stamp: "Formatted at 2026-08-09 02:15:59 on dist-test-slave-s11t"
server_key: "c702da4bc8467c950649298fe470606a"
server_key_iv: "2898f9412f331c3c1c60140d24167f42"
server_key_version: "encryptionkey@0"
I20260809 02:15:59.488569 19209 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:15:59.509994 19209 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:15:59.512720 19209 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:15:59.512929 19209 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:15:59.520229 19209 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.190:34113
I20260809 02:15:59.520259 19310 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.190:34113 every 8 connection(s)
I20260809 02:15:59.521162 19209 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
I20260809 02:15:59.523823 19311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:15:59.528841 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 19209
I20260809 02:15:59.528934 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/minicluster-data/master-0/wal/instance
I20260809 02:15:59.529209 15662 external_mini_cluster.cc:1428] Setting key ed28f061e26c56bf2c6303a5ce5a4a40
I20260809 02:15:59.529311 19311 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 512f28f7969f47b195c73785aad771ab: Bootstrap starting.
I20260809 02:15:59.531989 19311 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 512f28f7969f47b195c73785aad771ab: Neither blocks nor log segments found. Creating new log.
I20260809 02:15:59.532924 19311 log.cc:824] T 00000000000000000000000000000000 P 512f28f7969f47b195c73785aad771ab: Log is configured to *not* fsync() on all Append() calls
I20260809 02:15:59.534965 19311 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 512f28f7969f47b195c73785aad771ab: No bootstrap required, opened a new log
Aug 09 02:15:59 dist-test-slave-s11t krb5kdc[19186](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241759, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:15:59.539499 19311 raft_consensus.cc:348] T 00000000000000000000000000000000 P 512f28f7969f47b195c73785aad771ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "512f28f7969f47b195c73785aad771ab" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 34113 } }
I20260809 02:15:59.539773 19311 raft_consensus.cc:374] T 00000000000000000000000000000000 P 512f28f7969f47b195c73785aad771ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:15:59.539837 19311 raft_consensus.cc:729] T 00000000000000000000000000000000 P 512f28f7969f47b195c73785aad771ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 512f28f7969f47b195c73785aad771ab, State: Initialized, Role: FOLLOWER
I20260809 02:15:59.540365 19311 consensus_queue.cc:260] T 00000000000000000000000000000000 P 512f28f7969f47b195c73785aad771ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "512f28f7969f47b195c73785aad771ab" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 34113 } }
I20260809 02:15:59.540544 19311 raft_consensus.cc:388] T 00000000000000000000000000000000 P 512f28f7969f47b195c73785aad771ab [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:15:59.540613 19311 raft_consensus.cc:482] T 00000000000000000000000000000000 P 512f28f7969f47b195c73785aad771ab [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:15:59.540733 19311 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 512f28f7969f47b195c73785aad771ab [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:15:59.541806 19311 raft_consensus.cc:504] T 00000000000000000000000000000000 P 512f28f7969f47b195c73785aad771ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "512f28f7969f47b195c73785aad771ab" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 34113 } }
I20260809 02:15:59.542160 19311 leader_election.cc:302] T 00000000000000000000000000000000 P 512f28f7969f47b195c73785aad771ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 512f28f7969f47b195c73785aad771ab; no voters:
I20260809 02:15:59.542507 19311 leader_election.cc:288] T 00000000000000000000000000000000 P 512f28f7969f47b195c73785aad771ab [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:15:59.542654 19316 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 512f28f7969f47b195c73785aad771ab [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:15:59.542939 19316 raft_consensus.cc:686] T 00000000000000000000000000000000 P 512f28f7969f47b195c73785aad771ab [term 1 LEADER]: Becoming Leader. State: Replica: 512f28f7969f47b195c73785aad771ab, State: Running, Role: LEADER
I20260809 02:15:59.543272 19316 consensus_queue.cc:237] T 00000000000000000000000000000000 P 512f28f7969f47b195c73785aad771ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "512f28f7969f47b195c73785aad771ab" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 34113 } }
I20260809 02:15:59.543843 19311 sys_catalog.cc:564] T 00000000000000000000000000000000 P 512f28f7969f47b195c73785aad771ab [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:15:59.544970 19314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:59.530528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36866 (local address 127.15.75.190:34113)
0809 02:15:59.530960 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:59.530978 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:59.531018 (+ 40us) server_negotiation.cc:407] Connection header received
0809 02:15:59.531802 (+ 784us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:59.531827 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:59.532251 (+ 424us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:59.532609 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:59.533567 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:59.534623 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:59.535781 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:59.536234 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:59.536358 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:59.539178 (+ 2820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:15:59.539184 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:15:59.539200 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:15:59.539230 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:15:59.541257 (+ 2027us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:59.541674 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:59.541683 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:59.541692 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:59.541836 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:59.542176 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:59.542181 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:59.542183 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:59.542739 (+ 556us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:15:59.543518 (+ 779us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:59.543664 (+ 146us) server_negotiation.cc:299] Negotiation successful
0809 02:15:59.543957 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":111,"threads_started":1}
I20260809 02:15:59.545014 19323 sys_catalog.cc:455] T 00000000000000000000000000000000 P 512f28f7969f47b195c73785aad771ab [sys.catalog]: SysCatalogTable state changed. Reason: New leader 512f28f7969f47b195c73785aad771ab. Latest consensus state: current_term: 1 leader_uuid: "512f28f7969f47b195c73785aad771ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "512f28f7969f47b195c73785aad771ab" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 34113 } } }
I20260809 02:15:59.545163 19323 sys_catalog.cc:458] T 00000000000000000000000000000000 P 512f28f7969f47b195c73785aad771ab [sys.catalog]: This master's current role is: LEADER
I20260809 02:15:59.545464 19317 sys_catalog.cc:455] T 00000000000000000000000000000000 P 512f28f7969f47b195c73785aad771ab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "512f28f7969f47b195c73785aad771ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "512f28f7969f47b195c73785aad771ab" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 34113 } } }
I20260809 02:15:59.545560 19317 sys_catalog.cc:458] T 00000000000000000000000000000000 P 512f28f7969f47b195c73785aad771ab [sys.catalog]: This master's current role is: LEADER
W20260809 02:15:59.548651 19342 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 512f28f7969f47b195c73785aad771ab: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:15:59.548775 19342 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:15:59.548846 19333 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:15:59.549516 19333 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:15:59.554288 19333 catalog_manager.cc:1347] Generated new cluster ID: e38ebc7196914266906be2b89038cfa8
I20260809 02:15:59.554348 19333 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:15:59.566673 19333 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:15:59.567816 19333 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:15:59.574050 19333 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 512f28f7969f47b195c73785aad771ab: Generated new TSK 0
I20260809 02:15:59.574746 19333 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.75.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab.
Entry for principal kudu/127.15.75.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab.
I20260809 02:15:59.607046 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.129:0
--local_ip_for_outbound_sockets=127.15.75.129
--webserver_interface=127.15.75.129
--webserver_port=0
--tserver_master_addrs=127.15.75.190:34113
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab
--principal=kudu/127.15.75.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:37085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:15:59.716493 19356 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:15:59.716837 19356 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:15:59.716917 19356 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:15:59.720875 19356 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:15:59.720958 19356 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:15:59.721050 19356 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.129
I20260809 02:15:59.726086 19356 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:37085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab
--principal=kudu/127.15.75.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.75.129
--webserver_port=0
--tserver_master_addrs=127.15.75.190:34113
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19356
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.129
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:15:59.727078 19356 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:15:59.728075 19356 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:15:59.737447 19364 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:15:59.737427 19365 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:15:59.737596 19356 server_base.cc:1034] running on GCE node
W20260809 02:15:59.737424 19369 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:15:59.738078 19356 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:15:59.738662 19356 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:15:59.739850 19356 hybrid_clock.cc:648] HybridClock initialized: now 1786241759739833 us; error 31 us; skew 500 ppm
Aug 09 02:15:59 dist-test-slave-s11t krb5kdc[19186](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241759, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:15:59.742542 19356 init.cc:377] Logged in from keytab as kudu/127.15.75.129@KRBTEST.COM (short username kudu)
I20260809 02:15:59.743352 19356 webserver.cc:460] Webserver started at http://127.15.75.129:34037/ using document root <none> and password file <none>
I20260809 02:15:59.743940 19356 fs_manager.cc:359] Metadata directory not provided
I20260809 02:15:59.744042 19356 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:15:59.744285 19356 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:15:59.746562 19356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-0/data/instance:
uuid: "7c7a290d1ad64f2c8513998dd48e34b4"
format_stamp: "Formatted at 2026-08-09 02:15:59 on dist-test-slave-s11t"
server_key: "c17288820e00e0c9e90b773ce7ddc468"
server_key_iv: "2594cb0ccb0810aeeea43cb34587f35f"
server_key_version: "encryptionkey@0"
I20260809 02:15:59.747117 19356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-0/wal/instance:
uuid: "7c7a290d1ad64f2c8513998dd48e34b4"
format_stamp: "Formatted at 2026-08-09 02:15:59 on dist-test-slave-s11t"
server_key: "c17288820e00e0c9e90b773ce7ddc468"
server_key_iv: "2594cb0ccb0810aeeea43cb34587f35f"
server_key_version: "encryptionkey@0"
I20260809 02:15:59.750641 19356 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260809 02:15:59.752921 19379 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:59.755164 19356 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 02:15:59.755292 19356 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-0/wal
uuid: "7c7a290d1ad64f2c8513998dd48e34b4"
format_stamp: "Formatted at 2026-08-09 02:15:59 on dist-test-slave-s11t"
server_key: "c17288820e00e0c9e90b773ce7ddc468"
server_key_iv: "2594cb0ccb0810aeeea43cb34587f35f"
server_key_version: "encryptionkey@0"
I20260809 02:15:59.755414 19356 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:15:59.775247 19356 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:15:59.778061 19356 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:15:59.778270 19356 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:15:59.778873 19356 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:15:59.779716 19356 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:15:59.779788 19356 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:59.779866 19356 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:15:59.779914 19356 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:59.789131 19356 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.129:37439
I20260809 02:15:59.789214 19556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.129:37439 every 8 connection(s)
I20260809 02:15:59.790063 19356 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
I20260809 02:15:59.793246 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 19356
I20260809 02:15:59.793388 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-0/wal/instance
I20260809 02:15:59.793751 15662 external_mini_cluster.cc:1428] Setting key eb58a2a8242acae3c3215d16cdf7ee42
Aug 09 02:15:59 dist-test-slave-s11t krb5kdc[19186](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241759, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.129@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.75.130@KRBTEST.COM; defaulting to no policy
I20260809 02:15:59.805428 19314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:59.791869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.129:33701 (local address 127.15.75.190:34113)
0809 02:15:59.792017 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:59.792023 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:59.793011 (+ 988us) server_negotiation.cc:407] Connection header received
0809 02:15:59.794606 (+ 1595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:59.794608 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:59.794651 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:59.794718 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:59.796176 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:59.796726 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:59.798125 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:59.798536 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:59.798660 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:59.801515 (+ 2855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:15:59.801518 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:15:59.801520 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:15:59.801543 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:15:59.803028 (+ 1485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:59.803711 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:59.803714 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:59.803716 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:59.803810 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:59.804225 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:59.804228 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:59.804229 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:59.804400 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:15:59.804538 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:59.805125 (+ 587us) server_negotiation.cc:299] Negotiation successful
0809 02:15:59.805251 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 02:15:59.806180 19560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:59.792313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:34113 (local address 127.15.75.129:33701)
0809 02:15:59.792866 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:59.792909 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:59.794390 (+ 1481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:59.794898 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:59.794907 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:59.795346 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:59.796021 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:59.796045 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:59.796854 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:59.796860 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:59.798001 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:59.798011 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:59.798677 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:59.798680 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:59.798817 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:59.799425 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:59.799459 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:59.801314 (+ 1855us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:15:59.803152 (+ 1838us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:15:59.803159 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:15:59.803172 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:15:59.803549 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:15:59.803928 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:15:59.803932 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:15:59.803935 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:15:59.804120 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:15:59.804553 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:15:59.804561 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:15:59.804890 (+ 329us) client_negotiation.cc:769] Sending connection context
0809 02:15:59.805084 (+ 194us) client_negotiation.cc:240] Negotiation successful
0809 02:15:59.805397 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":141,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.130@KRBTEST.COM" created.
I20260809 02:15:59.807555 19557 heartbeater.cc:347] Connected to a master server at 127.15.75.190:34113
I20260809 02:15:59.807814 19557 heartbeater.cc:464] Registering TS with master...
I20260809 02:15:59.808301 19557 heartbeater.cc:511] Master 127.15.75.190:34113 requested a full tablet report, sending...
I20260809 02:15:59.809826 19255 ts_manager.cc:194] Registered new tserver with Master: 7c7a290d1ad64f2c8513998dd48e34b4 (127.15.75.129:37439)
I20260809 02:15:59.810932 19255 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.75.129@KRBTEST.COM'} at 127.15.75.129:33701
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab.
Entry for principal kudu/127.15.75.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab.
I20260809 02:15:59.821535 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.130:0
--local_ip_for_outbound_sockets=127.15.75.130
--webserver_interface=127.15.75.130
--webserver_port=0
--tserver_master_addrs=127.15.75.190:34113
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab
--principal=kudu/127.15.75.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:37085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:15:59.931907 19563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:15:59.932214 19563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:15:59.932269 19563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:15:59.937320 19563 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:15:59.937373 19563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:15:59.937429 19563 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.130
I20260809 02:15:59.942375 19563 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:37085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab
--principal=kudu/127.15.75.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.75.130
--webserver_port=0
--tserver_master_addrs=127.15.75.190:34113
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19563
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.130
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:15:59.943329 19563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:15:59.944320 19563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:15:59.953706 19571 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:15:59.953732 19570 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:15:59.953755 19573 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:15:59.953934 19563 server_base.cc:1034] running on GCE node
I20260809 02:15:59.954298 19563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:15:59.954912 19563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:15:59.956097 19563 hybrid_clock.cc:648] HybridClock initialized: now 1786241759956084 us; error 25 us; skew 500 ppm
Aug 09 02:15:59 dist-test-slave-s11t krb5kdc[19186](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241759, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:15:59.958902 19563 init.cc:377] Logged in from keytab as kudu/127.15.75.130@KRBTEST.COM (short username kudu)
I20260809 02:15:59.959746 19563 webserver.cc:460] Webserver started at http://127.15.75.130:45215/ using document root <none> and password file <none>
I20260809 02:15:59.960281 19563 fs_manager.cc:359] Metadata directory not provided
I20260809 02:15:59.960371 19563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:15:59.960598 19563 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:15:59.962503 19563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-1/data/instance:
uuid: "2cf415037df94c21bd8ec051ad857d74"
format_stamp: "Formatted at 2026-08-09 02:15:59 on dist-test-slave-s11t"
server_key: "8bbb792522b5b302d336868d79dabb58"
server_key_iv: "a6696ab9eed218e52510256391da032f"
server_key_version: "encryptionkey@0"
I20260809 02:15:59.963083 19563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-1/wal/instance:
uuid: "2cf415037df94c21bd8ec051ad857d74"
format_stamp: "Formatted at 2026-08-09 02:15:59 on dist-test-slave-s11t"
server_key: "8bbb792522b5b302d336868d79dabb58"
server_key_iv: "a6696ab9eed218e52510256391da032f"
server_key_version: "encryptionkey@0"
I20260809 02:15:59.966540 19563 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 02:15:59.968897 19584 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:59.970039 19563 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 02:15:59.970170 19563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-1/wal
uuid: "2cf415037df94c21bd8ec051ad857d74"
format_stamp: "Formatted at 2026-08-09 02:15:59 on dist-test-slave-s11t"
server_key: "8bbb792522b5b302d336868d79dabb58"
server_key_iv: "a6696ab9eed218e52510256391da032f"
server_key_version: "encryptionkey@0"
I20260809 02:15:59.970278 19563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:15:59.990351 19563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:15:59.993592 19563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:15:59.993791 19563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:15:59.994370 19563 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:15:59.995182 19563 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:15:59.995252 19563 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:59.995324 19563 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:15:59.995375 19563 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:00.005432 19563 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.130:43511
I20260809 02:16:00.005470 19753 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.130:43511 every 8 connection(s)
I20260809 02:16:00.006299 19563 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
I20260809 02:16:00.007442 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 19563
I20260809 02:16:00.007555 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-1/wal/instance
I20260809 02:16:00.007874 15662 external_mini_cluster.cc:1428] Setting key a191530f089f9928f91caca753f09172
Aug 09 02:16:00 dist-test-slave-s11t krb5kdc[19186](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241759, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.130@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.75.131@KRBTEST.COM; defaulting to no policy
I20260809 02:16:00.020578 19314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:00.008230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.130:50255 (local address 127.15.75.190:34113)
0809 02:16:00.008357 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:00.008362 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:00.009501 (+ 1139us) server_negotiation.cc:407] Connection header received
0809 02:16:00.010912 (+ 1411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:00.010915 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:00.010971 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:00.011074 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:00.012185 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:00.012778 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:00.013921 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:00.014145 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:00.014249 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:00.016724 (+ 2475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:00.016730 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:00.016734 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:00.016765 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:00.018582 (+ 1817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:00.019122 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:00.019125 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:00.019127 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:00.019213 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:00.019511 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:00.019513 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:00.019514 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:00.019754 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:00.019847 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:00.020349 (+ 502us) server_negotiation.cc:299] Negotiation successful
0809 02:16:00.020458 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 02:16:00.021347 19759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:00.008738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:34113 (local address 127.15.75.130:50255)
0809 02:16:00.009357 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:00.009421 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:00.010695 (+ 1274us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:00.011247 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:00.011255 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:00.011602 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:00.012046 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:00.012059 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:00.012898 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:00.012901 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:00.013782 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:00.013791 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:00.014297 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:00.014302 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:00.014483 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:00.015070 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:00.015087 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:00.016532 (+ 1445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:16:00.018703 (+ 2171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:00.018707 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:00.018718 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:00.019020 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:00.019300 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:00.019302 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:00.019303 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:00.019430 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:00.019871 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:16:00.019880 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:16:00.020118 (+ 238us) client_negotiation.cc:769] Sending connection context
0809 02:16:00.020273 (+ 155us) client_negotiation.cc:240] Negotiation successful
0809 02:16:00.020508 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":144,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.131@KRBTEST.COM" created.
I20260809 02:16:00.022504 19756 heartbeater.cc:347] Connected to a master server at 127.15.75.190:34113
I20260809 02:16:00.022739 19756 heartbeater.cc:464] Registering TS with master...
I20260809 02:16:00.023211 19756 heartbeater.cc:511] Master 127.15.75.190:34113 requested a full tablet report, sending...
I20260809 02:16:00.024267 19255 ts_manager.cc:194] Registered new tserver with Master: 2cf415037df94c21bd8ec051ad857d74 (127.15.75.130:43511)
I20260809 02:16:00.025121 19255 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.75.130@KRBTEST.COM'} at 127.15.75.130:50255
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab.
Entry for principal kudu/127.15.75.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab.
I20260809 02:16:00.037077 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.131:0
--local_ip_for_outbound_sockets=127.15.75.131
--webserver_interface=127.15.75.131
--webserver_port=0
--tserver_master_addrs=127.15.75.190:34113
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab
--principal=kudu/127.15.75.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:37085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:16:00.148535 19763 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:16:00.148833 19763 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:16:00.148916 19763 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:16:00.152839 19763 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:16:00.152911 19763 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:16:00.153010 19763 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.131
I20260809 02:16:00.162503 19763 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:37085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab
--principal=kudu/127.15.75.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.75.131
--webserver_port=0
--tserver_master_addrs=127.15.75.190:34113
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19763
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.131
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:16:00.163426 19763 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:16:00.164464 19763 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:16:00.177054 19773 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:00.177058 19769 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:00.177103 19768 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:00.177759 19763 server_base.cc:1034] running on GCE node
I20260809 02:16:00.178243 19763 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:16:00.179080 19763 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:16:00.180287 19763 hybrid_clock.cc:648] HybridClock initialized: now 1786241760180250 us; error 51 us; skew 500 ppm
Aug 09 02:16:00 dist-test-slave-s11t krb5kdc[19186](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241760, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:16:00.183528 19763 init.cc:377] Logged in from keytab as kudu/127.15.75.131@KRBTEST.COM (short username kudu)
I20260809 02:16:00.185213 19763 webserver.cc:460] Webserver started at http://127.15.75.131:42975/ using document root <none> and password file <none>
I20260809 02:16:00.185909 19763 fs_manager.cc:359] Metadata directory not provided
I20260809 02:16:00.185999 19763 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:16:00.186230 19763 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:16:00.188879 19763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-2/data/instance:
uuid: "a734d88df0394e3b95d425e5c92c0c18"
format_stamp: "Formatted at 2026-08-09 02:16:00 on dist-test-slave-s11t"
server_key: "683304b3886d56842de47b71bc83eaad"
server_key_iv: "6d0d5d4561c6e7f5d0eee5cd46b7c731"
server_key_version: "encryptionkey@0"
I20260809 02:16:00.189553 19763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-2/wal/instance:
uuid: "a734d88df0394e3b95d425e5c92c0c18"
format_stamp: "Formatted at 2026-08-09 02:16:00 on dist-test-slave-s11t"
server_key: "683304b3886d56842de47b71bc83eaad"
server_key_iv: "6d0d5d4561c6e7f5d0eee5cd46b7c731"
server_key_version: "encryptionkey@0"
I20260809 02:16:00.194453 19763 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 02:16:00.197955 19782 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:00.199311 19763 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 02:16:00.199437 19763 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-2/wal
uuid: "a734d88df0394e3b95d425e5c92c0c18"
format_stamp: "Formatted at 2026-08-09 02:16:00 on dist-test-slave-s11t"
server_key: "683304b3886d56842de47b71bc83eaad"
server_key_iv: "6d0d5d4561c6e7f5d0eee5cd46b7c731"
server_key_version: "encryptionkey@0"
I20260809 02:16:00.199553 19763 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:16:00.210310 19763 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:16:00.213370 19763 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:16:00.213569 19763 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:16:00.214151 19763 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:16:00.215059 19763 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:16:00.215126 19763 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:00.215188 19763 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:16:00.215220 19763 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:00.224282 19763 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.131:45641
I20260809 02:16:00.224328 19949 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.131:45641 every 8 connection(s)
I20260809 02:16:00.225236 19763 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
Aug 09 02:16:00 dist-test-slave-s11t krb5kdc[19186](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241760, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.131@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:16:00.234879 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 19763
I20260809 02:16:00.234993 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-2/wal/instance
I20260809 02:16:00.235337 15662 external_mini_cluster.cc:1428] Setting key 42192e99a2477cae07ce515b96a9c087
I20260809 02:16:00.238157 19314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:00.226932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.131:34155 (local address 127.15.75.190:34113)
0809 02:16:00.227066 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:00.227070 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:00.227809 (+ 739us) server_negotiation.cc:407] Connection header received
0809 02:16:00.229073 (+ 1264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:00.229077 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:00.229125 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:00.229201 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:00.230376 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:00.230908 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:00.232025 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:00.232235 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:00.232312 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:00.234567 (+ 2255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:00.234570 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:00.234572 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:00.234594 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:00.236181 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:00.236711 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:00.236715 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:00.236717 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:00.236809 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:00.237142 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:00.237145 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:00.237146 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:00.237380 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:00.237485 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:00.237924 (+ 439us) server_negotiation.cc:299] Negotiation successful
0809 02:16:00.238036 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 02:16:00.238808 19952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:00.227244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:34113 (local address 127.15.75.131:34155)
0809 02:16:00.227657 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:00.227695 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:00.228870 (+ 1175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:00.229445 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:00.229452 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:00.229793 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:00.230247 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:00.230260 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:00.231016 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:00.231018 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:00.231917 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:00.231926 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:00.232369 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:00.232372 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:00.232516 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:00.233016 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:00.233032 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:00.234415 (+ 1383us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:16:00.236302 (+ 1887us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:00.236308 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:00.236322 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:00.236568 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:00.236909 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:00.236912 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:00.236913 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:00.237044 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:00.237473 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:16:00.237479 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:16:00.237689 (+ 210us) client_negotiation.cc:769] Sending connection context
0809 02:16:00.237838 (+ 149us) client_negotiation.cc:240] Negotiation successful
0809 02:16:00.238057 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":117,"threads_started":1}
I20260809 02:16:00.239805 19950 heartbeater.cc:347] Connected to a master server at 127.15.75.190:34113
I20260809 02:16:00.240042 19950 heartbeater.cc:464] Registering TS with master...
I20260809 02:16:00.240494 19950 heartbeater.cc:511] Master 127.15.75.190:34113 requested a full tablet report, sending...
I20260809 02:16:00.241501 19255 ts_manager.cc:194] Registered new tserver with Master: a734d88df0394e3b95d425e5c92c0c18 (127.15.75.131:45641)
I20260809 02:16:00.242075 19255 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.75.131@KRBTEST.COM'} at 127.15.75.131:34155
I20260809 02:16:00.249379 15662 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 02:16:00.265228 19314 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:16:00.263758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36872 (local address 127.15.75.190:34113)
0809 02:16:00.263905 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:00.263909 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:00.264092 (+ 183us) server_negotiation.cc:407] Connection header received
0809 02:16:00.264189 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:00.264192 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:00.264229 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:00.264311 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:00.265058 (+ 747us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36872: BlockingRecv error: recv got EOF from 127.0.0.1:36872 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 02:16:00.266987 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 19356
I20260809 02:16:00.295475 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 19563
I20260809 02:16:00.303057 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 19763
I20260809 02:16:00.313256 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 19209
2026-08-09T02:16:00Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2341 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241741788589-15662-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:16:00 dist-test-slave-s11t krb5kdc[19964](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:16:00 dist-test-slave-s11t krb5kdc[19964](info): set up 2 sockets
Aug 09 02:16:00 dist-test-slave-s11t krb5kdc[19964](info): commencing operation
krb5kdc: starting...
W20260809 02:16:01.552956 15662 mini_kdc.cc:121] Time spent starting KDC: real 1.212s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Aug 09 02:16:01 dist-test-slave-s11t krb5kdc[19964](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241761, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:16:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:16:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.75.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab.
Entry for principal kudu/127.15.75.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab.
I20260809 02:16:01.677903 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241741788589-15662-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241741788589-15662-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241741788589-15662-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:40467
--webserver_interface=127.15.75.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab
--principal=kudu/127.15.75.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:41409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.75.190:40467
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:16:01.789907 19982 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:16:01.790153 19982 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:16:01.790244 19982 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:16:01.794420 19982 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:16:01.794476 19982 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:16:01.794497 19982 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:16:01.794549 19982 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:16:01.794588 19982 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:16:01.799916 19982 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:41409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241741788589-15662-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241741788589-15662-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.75.190:40467
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab
--principal=kudu/127.15.75.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:40467
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.75.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19982
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241741788589-15662-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:16:01.801012 19982 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:16:01.802057 19982 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:16:01.810693 19991 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:01.810680 19995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:01.810703 19990 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:01.810756 19982 server_base.cc:1034] running on GCE node
I20260809 02:16:01.811514 19982 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:16:01.812588 19982 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:16:01.813771 19982 hybrid_clock.cc:648] HybridClock initialized: now 1786241761813758 us; error 25 us; skew 500 ppm
Aug 09 02:16:01 dist-test-slave-s11t krb5kdc[19964](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241761, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:16:01.816573 19982 init.cc:377] Logged in from keytab as kudu/127.15.75.190@KRBTEST.COM (short username kudu)
I20260809 02:16:01.817456 19982 webserver.cc:460] Webserver started at http://127.15.75.190:39491/ using document root <none> and password file <none>
I20260809 02:16:01.818015 19982 fs_manager.cc:359] Metadata directory not provided
I20260809 02:16:01.818107 19982 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:16:01.818341 19982 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:16:01.820370 19982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241741788589-15662-0/minicluster-data/master-0/data/instance:
uuid: "716230ed4d5d41278dd6c01b609a2b15"
format_stamp: "Formatted at 2026-08-09 02:16:01 on dist-test-slave-s11t"
server_key: "29ee9b05ec2fce50b4ee275d3403aae5"
server_key_iv: "6fb503675764bd18e5ec0c09f05b7d9e"
server_key_version: "encryptionkey@0"
I20260809 02:16:01.820971 19982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241741788589-15662-0/minicluster-data/master-0/wal/instance:
uuid: "716230ed4d5d41278dd6c01b609a2b15"
format_stamp: "Formatted at 2026-08-09 02:16:01 on dist-test-slave-s11t"
server_key: "29ee9b05ec2fce50b4ee275d3403aae5"
server_key_iv: "6fb503675764bd18e5ec0c09f05b7d9e"
server_key_version: "encryptionkey@0"
I20260809 02:16:01.824622 19982 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260809 02:16:01.826961 20005 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:01.828116 19982 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 02:16:01.828269 19982 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241741788589-15662-0/minicluster-data/master-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241741788589-15662-0/minicluster-data/master-0/wal
uuid: "716230ed4d5d41278dd6c01b609a2b15"
format_stamp: "Formatted at 2026-08-09 02:16:01 on dist-test-slave-s11t"
server_key: "29ee9b05ec2fce50b4ee275d3403aae5"
server_key_iv: "6fb503675764bd18e5ec0c09f05b7d9e"
server_key_version: "encryptionkey@0"
I20260809 02:16:01.828379 19982 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241741788589-15662-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241741788589-15662-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241741788589-15662-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:16:01.863490 19982 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:16:01.867105 19982 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:16:01.867357 19982 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:16:01.874550 19982 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.190:40467
I20260809 02:16:01.874845 20087 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.190:40467 every 8 connection(s)
I20260809 02:16:01.875597 19982 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
I20260809 02:16:01.878508 20088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:16:01.883816 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 19982
I20260809 02:16:01.883963 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241741788589-15662-0/minicluster-data/master-0/wal/instance
I20260809 02:16:01.884092 20088 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 716230ed4d5d41278dd6c01b609a2b15: Bootstrap starting.
I20260809 02:16:01.884286 15662 external_mini_cluster.cc:1428] Setting key 03c4b12fc605e47a9ec40d771e2980cf
I20260809 02:16:01.886566 20088 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 716230ed4d5d41278dd6c01b609a2b15: Neither blocks nor log segments found. Creating new log.
I20260809 02:16:01.887331 20088 log.cc:824] T 00000000000000000000000000000000 P 716230ed4d5d41278dd6c01b609a2b15: Log is configured to *not* fsync() on all Append() calls
I20260809 02:16:01.889447 20088 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 716230ed4d5d41278dd6c01b609a2b15: No bootstrap required, opened a new log
Aug 09 02:16:01 dist-test-slave-s11t krb5kdc[19964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241761, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:16:01.894071 20088 raft_consensus.cc:348] T 00000000000000000000000000000000 P 716230ed4d5d41278dd6c01b609a2b15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "716230ed4d5d41278dd6c01b609a2b15" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 40467 } }
I20260809 02:16:01.894312 20088 raft_consensus.cc:374] T 00000000000000000000000000000000 P 716230ed4d5d41278dd6c01b609a2b15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:16:01.894373 20088 raft_consensus.cc:729] T 00000000000000000000000000000000 P 716230ed4d5d41278dd6c01b609a2b15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 716230ed4d5d41278dd6c01b609a2b15, State: Initialized, Role: FOLLOWER
I20260809 02:16:01.894807 20088 consensus_queue.cc:260] T 00000000000000000000000000000000 P 716230ed4d5d41278dd6c01b609a2b15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "716230ed4d5d41278dd6c01b609a2b15" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 40467 } }
I20260809 02:16:01.894982 20088 raft_consensus.cc:388] T 00000000000000000000000000000000 P 716230ed4d5d41278dd6c01b609a2b15 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:16:01.895078 20088 raft_consensus.cc:482] T 00000000000000000000000000000000 P 716230ed4d5d41278dd6c01b609a2b15 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:16:01.895195 20088 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 716230ed4d5d41278dd6c01b609a2b15 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:16:01.896384 20088 raft_consensus.cc:504] T 00000000000000000000000000000000 P 716230ed4d5d41278dd6c01b609a2b15 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "716230ed4d5d41278dd6c01b609a2b15" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 40467 } }
I20260809 02:16:01.896788 20088 leader_election.cc:302] T 00000000000000000000000000000000 P 716230ed4d5d41278dd6c01b609a2b15 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 716230ed4d5d41278dd6c01b609a2b15; no voters:
I20260809 02:16:01.897126 20088 leader_election.cc:288] T 00000000000000000000000000000000 P 716230ed4d5d41278dd6c01b609a2b15 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:16:01.897238 20094 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 716230ed4d5d41278dd6c01b609a2b15 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:16:01.897507 20094 raft_consensus.cc:686] T 00000000000000000000000000000000 P 716230ed4d5d41278dd6c01b609a2b15 [term 1 LEADER]: Becoming Leader. State: Replica: 716230ed4d5d41278dd6c01b609a2b15, State: Running, Role: LEADER
I20260809 02:16:01.897899 20094 consensus_queue.cc:237] T 00000000000000000000000000000000 P 716230ed4d5d41278dd6c01b609a2b15 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "716230ed4d5d41278dd6c01b609a2b15" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 40467 } }
I20260809 02:16:01.898365 20088 sys_catalog.cc:564] T 00000000000000000000000000000000 P 716230ed4d5d41278dd6c01b609a2b15 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:16:01.899158 20094 sys_catalog.cc:455] T 00000000000000000000000000000000 P 716230ed4d5d41278dd6c01b609a2b15 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 716230ed4d5d41278dd6c01b609a2b15. Latest consensus state: current_term: 1 leader_uuid: "716230ed4d5d41278dd6c01b609a2b15" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "716230ed4d5d41278dd6c01b609a2b15" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 40467 } } }
I20260809 02:16:01.899313 20094 sys_catalog.cc:458] T 00000000000000000000000000000000 P 716230ed4d5d41278dd6c01b609a2b15 [sys.catalog]: This master's current role is: LEADER
I20260809 02:16:01.899770 20095 sys_catalog.cc:455] T 00000000000000000000000000000000 P 716230ed4d5d41278dd6c01b609a2b15 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "716230ed4d5d41278dd6c01b609a2b15" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "716230ed4d5d41278dd6c01b609a2b15" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 40467 } } }
I20260809 02:16:01.899926 20095 sys_catalog.cc:458] T 00000000000000000000000000000000 P 716230ed4d5d41278dd6c01b609a2b15 [sys.catalog]: This master's current role is: LEADER
I20260809 02:16:01.900170 20092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:01.885667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55550 (local address 127.15.75.190:40467)
0809 02:16:01.886104 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:01.886119 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:01.886157 (+ 38us) server_negotiation.cc:407] Connection header received
0809 02:16:01.886847 (+ 690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:01.886865 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:01.887199 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:01.887510 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:01.888501 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:01.889574 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:01.890713 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:01.891178 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:01.891305 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:01.894392 (+ 3087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:01.894402 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:01.894420 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:01.894472 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:01.896548 (+ 2076us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:01.897058 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:01.897067 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:01.897076 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:01.897225 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:01.897603 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:01.897609 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:01.897611 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:01.898143 (+ 532us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:01.898314 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:01.898794 (+ 480us) server_negotiation.cc:299] Negotiation successful
0809 02:16:01.899096 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268,"thread_start_us":136,"threads_started":1}
I20260809 02:16:01.903524 15662 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260809 02:16:01.903787 20115 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 716230ed4d5d41278dd6c01b609a2b15: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:16:01.903877 20115 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:16:01.903991 20102 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:16:01.904760 20102 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:16:01.910768 20102 catalog_manager.cc:1347] Generated new cluster ID: a25804abfa714c3e8d757bb65465725d
I20260809 02:16:01.910830 20102 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:16:01.932564 20092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:01.926594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55552 (local address 127.15.75.190:40467)
0809 02:16:01.926732 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:01.926737 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:01.927062 (+ 325us) server_negotiation.cc:407] Connection header received
0809 02:16:01.927218 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:01.927222 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:01.927281 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:01.927368 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:01.927995 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:01.928814 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:01.929987 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:01.930344 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:01.930434 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:01.931397 (+ 963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:01.931401 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:01.931404 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:16:01.932073 (+ 669us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:01.932109 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:16:01.932128 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:01.932210 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:01.932339 (+ 129us) server_negotiation.cc:299] Negotiation successful
0809 02:16:01.932405 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 02:16:01.953151 20102 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:16:01.954586 20102 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:16:01.960645 20102 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 716230ed4d5d41278dd6c01b609a2b15: Generated new TSK 0
I20260809 02:16:01.961493 20102 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:16:01.970518 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 19982
2026-08-09T02:16:01Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1658 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:16:02 dist-test-slave-s11t krb5kdc[20131](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:16:02 dist-test-slave-s11t krb5kdc[20131](info): set up 2 sockets
Aug 09 02:16:02 dist-test-slave-s11t krb5kdc[20131](info): commencing operation
krb5kdc: starting...
W20260809 02:16:03.210263 15662 mini_kdc.cc:121] Time spent starting KDC: real 1.211s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Aug 09 02:16:03 dist-test-slave-s11t krb5kdc[20131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241763, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:16:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:16:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.75.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab.
Entry for principal kudu/127.15.75.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab.
I20260809 02:16:03.331501 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:39071
--webserver_interface=127.15.75.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab
--principal=kudu/127.15.75.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:35499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.75.190:39071
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:16:03.442466 20149 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:16:03.442718 20149 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:16:03.442759 20149 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:16:03.446789 20149 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:16:03.446842 20149 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:16:03.446862 20149 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:16:03.446879 20149 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:16:03.446894 20149 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:16:03.452008 20149 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:35499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.75.190:39071
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab
--principal=kudu/127.15.75.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:39071
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.75.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20149
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:16:03.452980 20149 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:16:03.453949 20149 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:16:03.462112 20156 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:03.462118 20157 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:03.462306 20149 server_base.cc:1034] running on GCE node
W20260809 02:16:03.462455 20159 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:03.462966 20149 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:16:03.464056 20149 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:16:03.465279 20149 hybrid_clock.cc:648] HybridClock initialized: now 1786241763465242 us; error 45 us; skew 500 ppm
Aug 09 02:16:03 dist-test-slave-s11t krb5kdc[20131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241763, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:16:03.472554 20149 init.cc:377] Logged in from keytab as kudu/127.15.75.190@KRBTEST.COM (short username kudu)
I20260809 02:16:03.473533 20149 webserver.cc:460] Webserver started at http://127.15.75.190:35907/ using document root <none> and password file <none>
I20260809 02:16:03.474089 20149 fs_manager.cc:359] Metadata directory not provided
I20260809 02:16:03.474184 20149 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:16:03.474432 20149 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:16:03.476449 20149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/minicluster-data/master-0/data/instance:
uuid: "d7020c365d5f4e9a94edb81d72117833"
format_stamp: "Formatted at 2026-08-09 02:16:03 on dist-test-slave-s11t"
server_key: "ea0ef0e0fca2888161993d93dfcb1a70"
server_key_iv: "786c2d4aff8d5947ad145d5b9a079a91"
server_key_version: "encryptionkey@0"
I20260809 02:16:03.477063 20149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/minicluster-data/master-0/wal/instance:
uuid: "d7020c365d5f4e9a94edb81d72117833"
format_stamp: "Formatted at 2026-08-09 02:16:03 on dist-test-slave-s11t"
server_key: "ea0ef0e0fca2888161993d93dfcb1a70"
server_key_iv: "786c2d4aff8d5947ad145d5b9a079a91"
server_key_version: "encryptionkey@0"
I20260809 02:16:03.480573 20149 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 02:16:03.482888 20167 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:03.484010 20149 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 02:16:03.484150 20149 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/minicluster-data/master-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/minicluster-data/master-0/wal
uuid: "d7020c365d5f4e9a94edb81d72117833"
format_stamp: "Formatted at 2026-08-09 02:16:03 on dist-test-slave-s11t"
server_key: "ea0ef0e0fca2888161993d93dfcb1a70"
server_key_iv: "786c2d4aff8d5947ad145d5b9a079a91"
server_key_version: "encryptionkey@0"
I20260809 02:16:03.484269 20149 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:16:03.495358 20149 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:16:03.498318 20149 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:16:03.498521 20149 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:16:03.505801 20149 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.190:39071
I20260809 02:16:03.505829 20237 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.190:39071 every 8 connection(s)
I20260809 02:16:03.506726 20149 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
I20260809 02:16:03.507349 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 20149
I20260809 02:16:03.507493 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/minicluster-data/master-0/wal/instance
I20260809 02:16:03.507784 15662 external_mini_cluster.cc:1428] Setting key c024dacad688a2ab4bb317b9f5e1305a
I20260809 02:16:03.510015 20239 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:16:03.515411 20239 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7020c365d5f4e9a94edb81d72117833: Bootstrap starting.
Aug 09 02:16:03 dist-test-slave-s11t krb5kdc[20131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241763, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:16:03.517714 20239 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d7020c365d5f4e9a94edb81d72117833: Neither blocks nor log segments found. Creating new log.
I20260809 02:16:03.518563 20239 log.cc:824] T 00000000000000000000000000000000 P d7020c365d5f4e9a94edb81d72117833: Log is configured to *not* fsync() on all Append() calls
I20260809 02:16:03.520787 20239 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7020c365d5f4e9a94edb81d72117833: No bootstrap required, opened a new log
I20260809 02:16:03.522318 20243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:03.509319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46146 (local address 127.15.75.190:39071)
0809 02:16:03.509764 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:03.509779 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:03.509814 (+ 35us) server_negotiation.cc:407] Connection header received
0809 02:16:03.510544 (+ 730us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:03.510562 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:03.510874 (+ 312us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:03.511193 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:03.512023 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:03.513089 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:03.513940 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:03.514407 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:03.514576 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:03.517253 (+ 2677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:03.517258 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:03.517270 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:03.517296 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:03.519203 (+ 1907us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:03.519736 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:03.519741 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:03.519745 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:03.519849 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:03.520165 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:03.520167 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:03.520168 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:03.520550 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:03.520785 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:03.521064 (+ 279us) server_negotiation.cc:299] Negotiation successful
0809 02:16:03.521378 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":136,"threads_started":1}
I20260809 02:16:03.523855 20239 raft_consensus.cc:348] T 00000000000000000000000000000000 P d7020c365d5f4e9a94edb81d72117833 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7020c365d5f4e9a94edb81d72117833" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 39071 } }
I20260809 02:16:03.524075 20239 raft_consensus.cc:374] T 00000000000000000000000000000000 P d7020c365d5f4e9a94edb81d72117833 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:16:03.524137 20239 raft_consensus.cc:729] T 00000000000000000000000000000000 P d7020c365d5f4e9a94edb81d72117833 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7020c365d5f4e9a94edb81d72117833, State: Initialized, Role: FOLLOWER
I20260809 02:16:03.524539 20239 consensus_queue.cc:260] T 00000000000000000000000000000000 P d7020c365d5f4e9a94edb81d72117833 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7020c365d5f4e9a94edb81d72117833" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 39071 } }
I20260809 02:16:03.524660 20239 raft_consensus.cc:388] T 00000000000000000000000000000000 P d7020c365d5f4e9a94edb81d72117833 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:16:03.524789 20239 raft_consensus.cc:482] T 00000000000000000000000000000000 P d7020c365d5f4e9a94edb81d72117833 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:16:03.524910 20239 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d7020c365d5f4e9a94edb81d72117833 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:16:03.525777 20239 raft_consensus.cc:504] T 00000000000000000000000000000000 P d7020c365d5f4e9a94edb81d72117833 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7020c365d5f4e9a94edb81d72117833" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 39071 } }
I20260809 02:16:03.526109 20239 leader_election.cc:302] T 00000000000000000000000000000000 P d7020c365d5f4e9a94edb81d72117833 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d7020c365d5f4e9a94edb81d72117833; no voters:
I20260809 02:16:03.526427 20239 leader_election.cc:288] T 00000000000000000000000000000000 P d7020c365d5f4e9a94edb81d72117833 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:16:03.526543 20245 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d7020c365d5f4e9a94edb81d72117833 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:16:03.526912 20245 raft_consensus.cc:686] T 00000000000000000000000000000000 P d7020c365d5f4e9a94edb81d72117833 [term 1 LEADER]: Becoming Leader. State: Replica: d7020c365d5f4e9a94edb81d72117833, State: Running, Role: LEADER
I20260809 02:16:03.527267 20245 consensus_queue.cc:237] T 00000000000000000000000000000000 P d7020c365d5f4e9a94edb81d72117833 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7020c365d5f4e9a94edb81d72117833" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 39071 } }
I20260809 02:16:03.527644 20239 sys_catalog.cc:564] T 00000000000000000000000000000000 P d7020c365d5f4e9a94edb81d72117833 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:16:03.528980 20247 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7020c365d5f4e9a94edb81d72117833 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d7020c365d5f4e9a94edb81d72117833. Latest consensus state: current_term: 1 leader_uuid: "d7020c365d5f4e9a94edb81d72117833" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7020c365d5f4e9a94edb81d72117833" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 39071 } } }
I20260809 02:16:03.529127 20247 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7020c365d5f4e9a94edb81d72117833 [sys.catalog]: This master's current role is: LEADER
I20260809 02:16:03.529428 20246 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7020c365d5f4e9a94edb81d72117833 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d7020c365d5f4e9a94edb81d72117833" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7020c365d5f4e9a94edb81d72117833" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 39071 } } }
I20260809 02:16:03.529513 20246 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7020c365d5f4e9a94edb81d72117833 [sys.catalog]: This master's current role is: LEADER
W20260809 02:16:03.532338 20265 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d7020c365d5f4e9a94edb81d72117833: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:16:03.532406 20265 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:16:03.532492 20256 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:16:03.533452 20256 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:16:03.539029 20256 catalog_manager.cc:1347] Generated new cluster ID: ca48ae0420a947ceb8f2f8e7b8b3f4fa
I20260809 02:16:03.539103 20256 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:16:03.549484 20256 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:16:03.550654 20256 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:16:03.557058 20256 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d7020c365d5f4e9a94edb81d72117833: Generated new TSK 0
I20260809 02:16:03.557801 20256 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.75.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab.
Entry for principal kudu/127.15.75.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab.
I20260809 02:16:03.585819 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.129:0
--local_ip_for_outbound_sockets=127.15.75.129
--webserver_interface=127.15.75.129
--webserver_port=0
--tserver_master_addrs=127.15.75.190:39071
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab
--principal=kudu/127.15.75.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:35499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:16:03.697137 20272 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:16:03.697395 20272 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:16:03.697497 20272 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:16:03.701534 20272 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:16:03.701581 20272 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:16:03.701663 20272 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.129
I20260809 02:16:03.706700 20272 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:35499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab
--principal=kudu/127.15.75.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.75.129
--webserver_port=0
--tserver_master_addrs=127.15.75.190:39071
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20272
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.129
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:16:03.707633 20272 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:16:03.708595 20272 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:16:03.717916 20285 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:03.717958 20281 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:03.718067 20272 server_base.cc:1034] running on GCE node
W20260809 02:16:03.718262 20282 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:03.718676 20272 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:16:03.719278 20272 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:16:03.720497 20272 hybrid_clock.cc:648] HybridClock initialized: now 1786241763720434 us; error 39 us; skew 500 ppm
Aug 09 02:16:03 dist-test-slave-s11t krb5kdc[20131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241763, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:16:03.723434 20272 init.cc:377] Logged in from keytab as kudu/127.15.75.129@KRBTEST.COM (short username kudu)
I20260809 02:16:03.724256 20272 webserver.cc:460] Webserver started at http://127.15.75.129:36309/ using document root <none> and password file <none>
I20260809 02:16:03.724854 20272 fs_manager.cc:359] Metadata directory not provided
I20260809 02:16:03.724931 20272 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:16:03.725108 20272 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:16:03.727260 20272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-0/data/instance:
uuid: "3118f5f4f0e54ef3b29b1ac45b102555"
format_stamp: "Formatted at 2026-08-09 02:16:03 on dist-test-slave-s11t"
server_key: "adc81627af56ad83d44a74f982a48f1b"
server_key_iv: "67bcc5e3695c8c4da70b7f4176a6dbff"
server_key_version: "encryptionkey@0"
I20260809 02:16:03.727861 20272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-0/wal/instance:
uuid: "3118f5f4f0e54ef3b29b1ac45b102555"
format_stamp: "Formatted at 2026-08-09 02:16:03 on dist-test-slave-s11t"
server_key: "adc81627af56ad83d44a74f982a48f1b"
server_key_iv: "67bcc5e3695c8c4da70b7f4176a6dbff"
server_key_version: "encryptionkey@0"
I20260809 02:16:03.731894 20272 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 02:16:03.734390 20294 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:03.735653 20272 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 02:16:03.735801 20272 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-0/wal
uuid: "3118f5f4f0e54ef3b29b1ac45b102555"
format_stamp: "Formatted at 2026-08-09 02:16:03 on dist-test-slave-s11t"
server_key: "adc81627af56ad83d44a74f982a48f1b"
server_key_iv: "67bcc5e3695c8c4da70b7f4176a6dbff"
server_key_version: "encryptionkey@0"
I20260809 02:16:03.735918 20272 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:16:03.749013 20272 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:16:03.751863 20272 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:16:03.752069 20272 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:16:03.752671 20272 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:16:03.753588 20272 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:16:03.753667 20272 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:03.753787 20272 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:16:03.753846 20272 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:03.763000 20272 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.129:33229
I20260809 02:16:03.763022 20453 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.129:33229 every 8 connection(s)
I20260809 02:16:03.763875 20272 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
I20260809 02:16:03.772542 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 20272
I20260809 02:16:03.772698 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-0/wal/instance
I20260809 02:16:03.773042 15662 external_mini_cluster.cc:1428] Setting key 87e23c0d857c87a9fe605ed3a88ea531
Aug 09 02:16:03 dist-test-slave-s11t krb5kdc[20131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241763, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.129@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:16:03.777935 20243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:03.765657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.129:54263 (local address 127.15.75.190:39071)
0809 02:16:03.765828 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:03.765833 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:03.766821 (+ 988us) server_negotiation.cc:407] Connection header received
0809 02:16:03.767674 (+ 853us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:03.767676 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:03.767722 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:03.767792 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:03.769086 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:03.769638 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:03.770843 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:03.771091 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:03.771169 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:03.773831 (+ 2662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:03.773835 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:03.773837 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:03.773859 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:03.775458 (+ 1599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:03.776095 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:03.776098 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:03.776099 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:03.776173 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:03.776569 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:03.776571 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:03.776572 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:03.776844 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:03.776986 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:03.777534 (+ 548us) server_negotiation.cc:299] Negotiation successful
0809 02:16:03.777681 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 02:16:03.778494 20456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:03.765987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:39071 (local address 127.15.75.129:54263)
0809 02:16:03.766633 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:03.766674 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:03.767492 (+ 818us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:03.767963 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:03.767970 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:03.768334 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:03.768940 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:03.768957 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:03.769778 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:03.769781 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:03.770733 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:03.770742 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:03.771225 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:03.771228 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:03.771368 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:03.771844 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:03.771861 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:03.773608 (+ 1747us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:16:03.775581 (+ 1973us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:03.775589 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:03.775605 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:03.775983 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:03.776266 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:03.776272 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:03.776276 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:03.776472 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:03.776995 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:16:03.777005 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:16:03.777284 (+ 279us) client_negotiation.cc:769] Sending connection context
0809 02:16:03.777451 (+ 167us) client_negotiation.cc:240] Negotiation successful
0809 02:16:03.777680 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":108,"threads_started":1}
I20260809 02:16:03.779608 20454 heartbeater.cc:347] Connected to a master server at 127.15.75.190:39071
I20260809 02:16:03.779887 20454 heartbeater.cc:464] Registering TS with master...
I20260809 02:16:03.780503 20454 heartbeater.cc:511] Master 127.15.75.190:39071 requested a full tablet report, sending...
I20260809 02:16:03.782063 20191 ts_manager.cc:194] Registered new tserver with Master: 3118f5f4f0e54ef3b29b1ac45b102555 (127.15.75.129:33229)
I20260809 02:16:03.783193 20191 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.75.129@KRBTEST.COM'} at 127.15.75.129:54263
WARNING: no policy specified for kudu/127.15.75.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab.
Entry for principal kudu/127.15.75.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab.
I20260809 02:16:03.804221 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.130:0
--local_ip_for_outbound_sockets=127.15.75.130
--webserver_interface=127.15.75.130
--webserver_port=0
--tserver_master_addrs=127.15.75.190:39071
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab
--principal=kudu/127.15.75.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:35499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:16:03.915385 20464 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:16:03.915606 20464 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:16:03.915645 20464 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:16:03.919539 20464 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:16:03.919590 20464 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:16:03.919644 20464 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.130
I20260809 02:16:03.924595 20464 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:35499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab
--principal=kudu/127.15.75.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.75.130
--webserver_port=0
--tserver_master_addrs=127.15.75.190:39071
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20464
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.130
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:16:03.925522 20464 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:16:03.926401 20464 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:16:03.935981 20475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:03.935942 20470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:03.935953 20472 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:03.936120 20464 server_base.cc:1034] running on GCE node
I20260809 02:16:03.936578 20464 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:16:03.937176 20464 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:16:03.938333 20464 hybrid_clock.cc:648] HybridClock initialized: now 1786241763938319 us; error 23 us; skew 500 ppm
Aug 09 02:16:03 dist-test-slave-s11t krb5kdc[20131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241763, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:16:03.941113 20464 init.cc:377] Logged in from keytab as kudu/127.15.75.130@KRBTEST.COM (short username kudu)
I20260809 02:16:03.941850 20464 webserver.cc:460] Webserver started at http://127.15.75.130:33457/ using document root <none> and password file <none>
I20260809 02:16:03.942345 20464 fs_manager.cc:359] Metadata directory not provided
I20260809 02:16:03.942405 20464 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:16:03.942556 20464 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:16:03.944480 20464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-1/data/instance:
uuid: "f66ffa795f3c45b48bd70706076b036a"
format_stamp: "Formatted at 2026-08-09 02:16:03 on dist-test-slave-s11t"
server_key: "586f9af7a17018943dd4670623ac6940"
server_key_iv: "8902ce211b46292bd0d904cc1cd77eaf"
server_key_version: "encryptionkey@0"
I20260809 02:16:03.945058 20464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-1/wal/instance:
uuid: "f66ffa795f3c45b48bd70706076b036a"
format_stamp: "Formatted at 2026-08-09 02:16:03 on dist-test-slave-s11t"
server_key: "586f9af7a17018943dd4670623ac6940"
server_key_iv: "8902ce211b46292bd0d904cc1cd77eaf"
server_key_version: "encryptionkey@0"
I20260809 02:16:03.948536 20464 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 02:16:03.950886 20486 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:03.952122 20464 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 02:16:03.952270 20464 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-1/wal
uuid: "f66ffa795f3c45b48bd70706076b036a"
format_stamp: "Formatted at 2026-08-09 02:16:03 on dist-test-slave-s11t"
server_key: "586f9af7a17018943dd4670623ac6940"
server_key_iv: "8902ce211b46292bd0d904cc1cd77eaf"
server_key_version: "encryptionkey@0"
I20260809 02:16:03.952378 20464 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:16:03.971266 20464 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:16:03.974498 20464 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:16:03.974704 20464 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:16:03.975301 20464 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:16:03.976122 20464 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:16:03.976171 20464 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:03.976239 20464 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:16:03.976274 20464 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:03.985735 20464 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.130:35383
I20260809 02:16:03.985760 20645 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.130:35383 every 8 connection(s)
I20260809 02:16:03.986585 20464 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
I20260809 02:16:03.990379 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 20464
I20260809 02:16:03.990504 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-1/wal/instance
I20260809 02:16:03.990826 15662 external_mini_cluster.cc:1428] Setting key 7245b0dd8b5a32be17fe4d2c0986436a
Aug 09 02:16:03 dist-test-slave-s11t krb5kdc[20131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241763, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.130@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.75.131@KRBTEST.COM; defaulting to no policy
I20260809 02:16:04.002561 20243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:03.988407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.130:53147 (local address 127.15.75.190:39071)
0809 02:16:03.988540 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:03.988547 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:03.989445 (+ 898us) server_negotiation.cc:407] Connection header received
0809 02:16:03.991273 (+ 1828us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:03.991277 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:03.991343 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:03.991432 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:03.992813 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:03.993336 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:03.994928 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:03.995379 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:03.995515 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:03.998839 (+ 3324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:03.998842 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:03.998844 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:03.998869 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:04.000385 (+ 1516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:04.000994 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:04.000997 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:04.000999 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:04.001082 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:04.001460 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:04.001464 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:04.001466 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:04.001650 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:04.001759 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:04.002263 (+ 504us) server_negotiation.cc:299] Negotiation successful
0809 02:16:04.002389 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 02:16:04.003360 20649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:03.988856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:39071 (local address 127.15.75.130:53147)
0809 02:16:03.989288 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:03.989328 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:03.991022 (+ 1694us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:03.991619 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:03.991628 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:03.991970 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:03.992630 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:03.992653 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:03.993468 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:03.993474 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:03.994788 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:03.994798 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:03.995758 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:03.995762 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:03.996011 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:03.996633 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:03.996650 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:03.998640 (+ 1990us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:16:04.000525 (+ 1885us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:04.000531 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:04.000545 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:04.000884 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:04.001201 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:04.001205 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:04.001207 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:04.001350 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:04.001773 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:16:04.001781 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:16:04.002021 (+ 240us) client_negotiation.cc:769] Sending connection context
0809 02:16:04.002202 (+ 181us) client_negotiation.cc:240] Negotiation successful
0809 02:16:04.002538 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":111,"threads_started":1}
I20260809 02:16:04.004643 20646 heartbeater.cc:347] Connected to a master server at 127.15.75.190:39071
I20260809 02:16:04.004992 20646 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.131@KRBTEST.COM" created.
I20260809 02:16:04.005599 20646 heartbeater.cc:511] Master 127.15.75.190:39071 requested a full tablet report, sending...
I20260809 02:16:04.006808 20191 ts_manager.cc:194] Registered new tserver with Master: f66ffa795f3c45b48bd70706076b036a (127.15.75.130:35383)
I20260809 02:16:04.007503 20191 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.75.130@KRBTEST.COM'} at 127.15.75.130:53147
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab.
Entry for principal kudu/127.15.75.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab.
I20260809 02:16:04.021525 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.131:0
--local_ip_for_outbound_sockets=127.15.75.131
--webserver_interface=127.15.75.131
--webserver_port=0
--tserver_master_addrs=127.15.75.190:39071
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab
--principal=kudu/127.15.75.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:35499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:16:04.136382 20656 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:16:04.136601 20656 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:16:04.136642 20656 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:16:04.140658 20656 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:16:04.140754 20656 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:16:04.140848 20656 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.131
I20260809 02:16:04.145859 20656 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:35499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab
--principal=kudu/127.15.75.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.75.131
--webserver_port=0
--tserver_master_addrs=127.15.75.190:39071
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20656
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.131
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:16:04.146781 20656 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:16:04.147727 20656 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:16:04.157354 20662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:04.157459 20656 server_base.cc:1034] running on GCE node
W20260809 02:16:04.157359 20663 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:04.157387 20667 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:04.158056 20656 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:16:04.158639 20656 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:16:04.159817 20656 hybrid_clock.cc:648] HybridClock initialized: now 1786241764159797 us; error 30 us; skew 500 ppm
Aug 09 02:16:04 dist-test-slave-s11t krb5kdc[20131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241764, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:16:04.162591 20656 init.cc:377] Logged in from keytab as kudu/127.15.75.131@KRBTEST.COM (short username kudu)
I20260809 02:16:04.163426 20656 webserver.cc:460] Webserver started at http://127.15.75.131:42927/ using document root <none> and password file <none>
I20260809 02:16:04.163919 20656 fs_manager.cc:359] Metadata directory not provided
I20260809 02:16:04.163975 20656 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:16:04.164130 20656 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:16:04.166071 20656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-2/data/instance:
uuid: "f394b0493a1b4bbe86adafefe4ee81ab"
format_stamp: "Formatted at 2026-08-09 02:16:04 on dist-test-slave-s11t"
server_key: "557979a5ba944cb33a0246a75b382339"
server_key_iv: "00a87b73e671b7e883941b8f21544356"
server_key_version: "encryptionkey@0"
I20260809 02:16:04.166565 20656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-2/wal/instance:
uuid: "f394b0493a1b4bbe86adafefe4ee81ab"
format_stamp: "Formatted at 2026-08-09 02:16:04 on dist-test-slave-s11t"
server_key: "557979a5ba944cb33a0246a75b382339"
server_key_iv: "00a87b73e671b7e883941b8f21544356"
server_key_version: "encryptionkey@0"
I20260809 02:16:04.170091 20656 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 02:16:04.172358 20680 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:04.173568 20656 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 02:16:04.173705 20656 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-2/wal
uuid: "f394b0493a1b4bbe86adafefe4ee81ab"
format_stamp: "Formatted at 2026-08-09 02:16:04 on dist-test-slave-s11t"
server_key: "557979a5ba944cb33a0246a75b382339"
server_key_iv: "00a87b73e671b7e883941b8f21544356"
server_key_version: "encryptionkey@0"
I20260809 02:16:04.173813 20656 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:16:04.190996 20656 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:16:04.194106 20656 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:16:04.194298 20656 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:16:04.194845 20656 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:16:04.195611 20656 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:16:04.195683 20656 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:04.195750 20656 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:16:04.195803 20656 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:04.205154 20656 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.131:36069
I20260809 02:16:04.205199 20847 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.131:36069 every 8 connection(s)
I20260809 02:16:04.205983 20656 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
I20260809 02:16:04.207878 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 20656
I20260809 02:16:04.207995 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241741788589-15662-0/minicluster-data/ts-2/wal/instance
I20260809 02:16:04.208312 15662 external_mini_cluster.cc:1428] Setting key 7f53538f90be669910286c8d71120913
Aug 09 02:16:04 dist-test-slave-s11t krb5kdc[20131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241764, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.131@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:16:04.219626 20243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:04.207601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.131:44821 (local address 127.15.75.190:39071)
0809 02:16:04.207764 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:04.207769 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:04.208660 (+ 891us) server_negotiation.cc:407] Connection header received
0809 02:16:04.210422 (+ 1762us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:04.210426 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:04.210482 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:04.210577 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:04.211751 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:04.212350 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:04.213539 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:04.213783 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:04.213871 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:04.216180 (+ 2309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:04.216183 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:04.216186 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:04.216213 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:04.217909 (+ 1696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:04.218335 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:04.218338 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:04.218339 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:04.218417 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:04.218727 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:04.218730 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:04.218731 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:04.218932 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:04.219017 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:04.219416 (+ 399us) server_negotiation.cc:299] Negotiation successful
0809 02:16:04.219512 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 02:16:04.220360 20850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:04.207998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:39071 (local address 127.15.75.131:44821)
0809 02:16:04.208496 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:04.208550 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:04.210186 (+ 1636us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:04.210751 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:04.210758 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:04.211092 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:04.211553 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:04.211565 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:04.212466 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:04.212469 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:04.213424 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:04.213434 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:04.213923 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:04.213925 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:04.214054 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:04.214552 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:04.214568 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:04.216028 (+ 1460us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:16:04.218006 (+ 1978us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:04.218010 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:04.218020 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:04.218247 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:04.218497 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:04.218499 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:04.218501 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:04.218617 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:04.219024 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:16:04.219028 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:16:04.219223 (+ 195us) client_negotiation.cc:769] Sending connection context
0809 02:16:04.219366 (+ 143us) client_negotiation.cc:240] Negotiation successful
0809 02:16:04.219583 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":100,"threads_started":1}
I20260809 02:16:04.221554 20848 heartbeater.cc:347] Connected to a master server at 127.15.75.190:39071
I20260809 02:16:04.221802 20848 heartbeater.cc:464] Registering TS with master...
I20260809 02:16:04.222357 20848 heartbeater.cc:511] Master 127.15.75.190:39071 requested a full tablet report, sending...
I20260809 02:16:04.223321 20191 ts_manager.cc:194] Registered new tserver with Master: f394b0493a1b4bbe86adafefe4ee81ab (127.15.75.131:36069)
I20260809 02:16:04.223878 20191 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.75.131@KRBTEST.COM'} at 127.15.75.131:44821
I20260809 02:16:04.233780 15662 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 02:16:04 dist-test-slave-s11t krb5kdc[20131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241764, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 09 02:16:04 dist-test-slave-s11t krb5kdc[20131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241764, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:16:04.269016 20243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:04.257060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46148 (local address 127.15.75.190:39071)
0809 02:16:04.257197 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:04.257202 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:04.257352 (+ 150us) server_negotiation.cc:407] Connection header received
0809 02:16:04.257523 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:04.257528 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:04.257590 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:04.257706 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:04.258647 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:04.259166 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:04.260527 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:04.260962 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:04.261087 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:04.264528 (+ 3441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:04.264535 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:04.264541 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:04.264571 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:04.267031 (+ 2460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:04.267527 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:04.267533 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:04.267535 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:04.267660 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:04.267964 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:04.267967 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:04.267968 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:04.268261 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:04.268416 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:04.268660 (+ 244us) server_negotiation.cc:299] Negotiation successful
0809 02:16:04.268836 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
W20260809 02:16:04.269443 20191 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:46148
I20260809 02:16:04.270393 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 20272
I20260809 02:16:04.292838 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 20464
I20260809 02:16:04.306124 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 20656
I20260809 02:16:04.315667 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 20149
2026-08-09T02:16:04Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2340 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:16:04 dist-test-slave-s11t krb5kdc[20863](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:16:04 dist-test-slave-s11t krb5kdc[20863](info): set up 2 sockets
Aug 09 02:16:04 dist-test-slave-s11t krb5kdc[20863](info): commencing operation
krb5kdc: starting...
W20260809 02:16:05.561712 15662 mini_kdc.cc:121] Time spent starting KDC: real 1.222s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Aug 09 02:16:05 dist-test-slave-s11t krb5kdc[20863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241765, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:16:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:16:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.75.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab.
Entry for principal kudu/127.15.75.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab.
I20260809 02:16:05.689347 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:44431
--webserver_interface=127.15.75.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab
--principal=kudu/127.15.75.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:41087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.75.190:44431
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:16:05.800504 20883 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:16:05.800772 20883 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:16:05.800812 20883 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:16:05.804821 20883 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:16:05.804872 20883 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:16:05.804898 20883 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:16:05.804914 20883 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:16:05.804929 20883 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:16:05.810108 20883 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:41087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.75.190:44431
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab
--principal=kudu/127.15.75.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:44431
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.75.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20883
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:16:05.811044 20883 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:16:05.811980 20883 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:16:05.820590 20894 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:05.820673 20883 server_base.cc:1034] running on GCE node
W20260809 02:16:05.820721 20892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:05.820730 20896 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:05.821497 20883 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:16:05.822597 20883 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:16:05.823798 20883 hybrid_clock.cc:648] HybridClock initialized: now 1786241765823781 us; error 32 us; skew 500 ppm
Aug 09 02:16:05 dist-test-slave-s11t krb5kdc[20863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241765, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:16:05.826804 20883 init.cc:377] Logged in from keytab as kudu/127.15.75.190@KRBTEST.COM (short username kudu)
I20260809 02:16:05.827669 20883 webserver.cc:460] Webserver started at http://127.15.75.190:34069/ using document root <none> and password file <none>
I20260809 02:16:05.828233 20883 fs_manager.cc:359] Metadata directory not provided
I20260809 02:16:05.828325 20883 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:16:05.828569 20883 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:16:05.830600 20883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/minicluster-data/master-0/data/instance:
uuid: "bbdb1d8b9d58447ebe32b2be725ff90f"
format_stamp: "Formatted at 2026-08-09 02:16:05 on dist-test-slave-s11t"
server_key: "537d9a58cf9c63ef4cc5c33783164e07"
server_key_iv: "8f5989eb0f09a1ef1581caa6fc5a398b"
server_key_version: "encryptionkey@0"
I20260809 02:16:05.831182 20883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/minicluster-data/master-0/wal/instance:
uuid: "bbdb1d8b9d58447ebe32b2be725ff90f"
format_stamp: "Formatted at 2026-08-09 02:16:05 on dist-test-slave-s11t"
server_key: "537d9a58cf9c63ef4cc5c33783164e07"
server_key_iv: "8f5989eb0f09a1ef1581caa6fc5a398b"
server_key_version: "encryptionkey@0"
I20260809 02:16:05.834885 20883 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 02:16:05.837226 20905 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:05.838394 20883 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 02:16:05.838536 20883 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/minicluster-data/master-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/minicluster-data/master-0/wal
uuid: "bbdb1d8b9d58447ebe32b2be725ff90f"
format_stamp: "Formatted at 2026-08-09 02:16:05 on dist-test-slave-s11t"
server_key: "537d9a58cf9c63ef4cc5c33783164e07"
server_key_iv: "8f5989eb0f09a1ef1581caa6fc5a398b"
server_key_version: "encryptionkey@0"
I20260809 02:16:05.838644 20883 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:16:05.850780 20883 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:16:05.853946 20883 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:16:05.854094 20883 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:16:05.860755 20883 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.190:44431
I20260809 02:16:05.860855 20992 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.190:44431 every 8 connection(s)
I20260809 02:16:05.861692 20883 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
I20260809 02:16:05.864780 20993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:16:05.865249 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 20883
I20260809 02:16:05.865391 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/minicluster-data/master-0/wal/instance
I20260809 02:16:05.865676 15662 external_mini_cluster.cc:1428] Setting key 7957b072e5b649c566efe91da93c642d
I20260809 02:16:05.872092 20993 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbdb1d8b9d58447ebe32b2be725ff90f: Bootstrap starting.
I20260809 02:16:05.874394 20993 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bbdb1d8b9d58447ebe32b2be725ff90f: Neither blocks nor log segments found. Creating new log.
Aug 09 02:16:05 dist-test-slave-s11t krb5kdc[20863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241765, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:16:05.875174 20993 log.cc:824] T 00000000000000000000000000000000 P bbdb1d8b9d58447ebe32b2be725ff90f: Log is configured to *not* fsync() on all Append() calls
I20260809 02:16:05.877056 20993 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbdb1d8b9d58447ebe32b2be725ff90f: No bootstrap required, opened a new log
I20260809 02:16:05.880105 20998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:05.867099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53016 (local address 127.15.75.190:44431)
0809 02:16:05.867529 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:05.867548 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:05.867588 (+ 40us) server_negotiation.cc:407] Connection header received
0809 02:16:05.868398 (+ 810us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:05.868423 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:05.868864 (+ 441us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:05.869256 (+ 392us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:05.869899 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:05.871415 (+ 1516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:05.872255 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:05.872815 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:05.873004 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:05.875514 (+ 2510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:05.875520 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:05.875533 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:05.875560 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:05.877371 (+ 1811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:05.877821 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:05.877827 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:05.877836 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:05.877945 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:05.878248 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:05.878251 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:05.878253 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:05.878632 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:05.878845 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:05.879140 (+ 295us) server_negotiation.cc:299] Negotiation successful
0809 02:16:05.879414 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":108,"threads_started":1}
I20260809 02:16:05.880633 20993 raft_consensus.cc:348] T 00000000000000000000000000000000 P bbdb1d8b9d58447ebe32b2be725ff90f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbdb1d8b9d58447ebe32b2be725ff90f" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 44431 } }
I20260809 02:16:05.880932 20993 raft_consensus.cc:374] T 00000000000000000000000000000000 P bbdb1d8b9d58447ebe32b2be725ff90f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:16:05.881027 20993 raft_consensus.cc:729] T 00000000000000000000000000000000 P bbdb1d8b9d58447ebe32b2be725ff90f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbdb1d8b9d58447ebe32b2be725ff90f, State: Initialized, Role: FOLLOWER
I20260809 02:16:05.881507 20993 consensus_queue.cc:260] T 00000000000000000000000000000000 P bbdb1d8b9d58447ebe32b2be725ff90f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbdb1d8b9d58447ebe32b2be725ff90f" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 44431 } }
I20260809 02:16:05.881670 20993 raft_consensus.cc:388] T 00000000000000000000000000000000 P bbdb1d8b9d58447ebe32b2be725ff90f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:16:05.881748 20993 raft_consensus.cc:482] T 00000000000000000000000000000000 P bbdb1d8b9d58447ebe32b2be725ff90f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:16:05.881846 20993 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bbdb1d8b9d58447ebe32b2be725ff90f [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:16:05.882874 20993 raft_consensus.cc:504] T 00000000000000000000000000000000 P bbdb1d8b9d58447ebe32b2be725ff90f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbdb1d8b9d58447ebe32b2be725ff90f" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 44431 } }
I20260809 02:16:05.883219 20993 leader_election.cc:302] T 00000000000000000000000000000000 P bbdb1d8b9d58447ebe32b2be725ff90f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bbdb1d8b9d58447ebe32b2be725ff90f; no voters:
I20260809 02:16:05.883553 20993 leader_election.cc:288] T 00000000000000000000000000000000 P bbdb1d8b9d58447ebe32b2be725ff90f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:16:05.883679 21001 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bbdb1d8b9d58447ebe32b2be725ff90f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:16:05.883935 21001 raft_consensus.cc:686] T 00000000000000000000000000000000 P bbdb1d8b9d58447ebe32b2be725ff90f [term 1 LEADER]: Becoming Leader. State: Replica: bbdb1d8b9d58447ebe32b2be725ff90f, State: Running, Role: LEADER
I20260809 02:16:05.884295 21001 consensus_queue.cc:237] T 00000000000000000000000000000000 P bbdb1d8b9d58447ebe32b2be725ff90f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbdb1d8b9d58447ebe32b2be725ff90f" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 44431 } }
I20260809 02:16:05.884892 20993 sys_catalog.cc:564] T 00000000000000000000000000000000 P bbdb1d8b9d58447ebe32b2be725ff90f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:16:05.886029 21004 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbdb1d8b9d58447ebe32b2be725ff90f [sys.catalog]: SysCatalogTable state changed. Reason: New leader bbdb1d8b9d58447ebe32b2be725ff90f. Latest consensus state: current_term: 1 leader_uuid: "bbdb1d8b9d58447ebe32b2be725ff90f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbdb1d8b9d58447ebe32b2be725ff90f" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 44431 } } }
I20260809 02:16:05.886153 21004 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbdb1d8b9d58447ebe32b2be725ff90f [sys.catalog]: This master's current role is: LEADER
I20260809 02:16:05.886467 21002 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbdb1d8b9d58447ebe32b2be725ff90f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bbdb1d8b9d58447ebe32b2be725ff90f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbdb1d8b9d58447ebe32b2be725ff90f" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 44431 } } }
I20260809 02:16:05.886570 21002 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbdb1d8b9d58447ebe32b2be725ff90f [sys.catalog]: This master's current role is: LEADER
I20260809 02:16:05.886642 21014 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:16:05.890084 21014 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:16:05.895526 21014 catalog_manager.cc:1347] Generated new cluster ID: 561453abe6784db79a7bd6be032054de
I20260809 02:16:05.895587 21014 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:16:05.918876 21014 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:16:05.920092 21014 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:16:05.934861 21014 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bbdb1d8b9d58447ebe32b2be725ff90f: Generated new TSK 0
I20260809 02:16:05.935642 21014 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.75.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab.
Entry for principal kudu/127.15.75.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab.
I20260809 02:16:05.967643 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.129:0
--local_ip_for_outbound_sockets=127.15.75.129
--webserver_interface=127.15.75.129
--webserver_port=0
--tserver_master_addrs=127.15.75.190:44431
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab
--principal=kudu/127.15.75.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:41087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:16:06.078553 21032 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:16:06.078828 21032 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:16:06.078914 21032 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:16:06.082940 21032 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:16:06.083014 21032 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:16:06.083102 21032 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.129
I20260809 02:16:06.088423 21032 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:41087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab
--principal=kudu/127.15.75.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.75.129
--webserver_port=0
--tserver_master_addrs=127.15.75.190:44431
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21032
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.129
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:16:06.089432 21032 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:16:06.090426 21032 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:16:06.100975 21040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:06.101025 21042 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:06.101018 21032 server_base.cc:1034] running on GCE node
W20260809 02:16:06.101168 21047 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:06.101558 21032 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:16:06.102118 21032 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:16:06.103277 21032 hybrid_clock.cc:648] HybridClock initialized: now 1786241766103247 us; error 41 us; skew 500 ppm
Aug 09 02:16:06 dist-test-slave-s11t krb5kdc[20863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241766, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:16:06.106153 21032 init.cc:377] Logged in from keytab as kudu/127.15.75.129@KRBTEST.COM (short username kudu)
I20260809 02:16:06.107008 21032 webserver.cc:460] Webserver started at http://127.15.75.129:41153/ using document root <none> and password file <none>
I20260809 02:16:06.107501 21032 fs_manager.cc:359] Metadata directory not provided
I20260809 02:16:06.107556 21032 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:16:06.107713 21032 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:16:06.109710 21032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/minicluster-data/ts-0/data/instance:
uuid: "4578b236c9cb4f4eba1b91e0372b3b75"
format_stamp: "Formatted at 2026-08-09 02:16:06 on dist-test-slave-s11t"
server_key: "d3078ffd52cc018fe3d340fde6ae92cb"
server_key_iv: "d3e32776cad7fe4f7d11e1e4c233f679"
server_key_version: "encryptionkey@0"
I20260809 02:16:06.110221 21032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/minicluster-data/ts-0/wal/instance:
uuid: "4578b236c9cb4f4eba1b91e0372b3b75"
format_stamp: "Formatted at 2026-08-09 02:16:06 on dist-test-slave-s11t"
server_key: "d3078ffd52cc018fe3d340fde6ae92cb"
server_key_iv: "d3e32776cad7fe4f7d11e1e4c233f679"
server_key_version: "encryptionkey@0"
I20260809 02:16:06.113644 21032 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 02:16:06.115830 21054 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:06.117053 21032 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 02:16:06.117172 21032 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/minicluster-data/ts-0/wal
uuid: "4578b236c9cb4f4eba1b91e0372b3b75"
format_stamp: "Formatted at 2026-08-09 02:16:06 on dist-test-slave-s11t"
server_key: "d3078ffd52cc018fe3d340fde6ae92cb"
server_key_iv: "d3e32776cad7fe4f7d11e1e4c233f679"
server_key_version: "encryptionkey@0"
I20260809 02:16:06.117269 21032 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:16:06.135767 21032 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:16:06.139031 21032 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:16:06.139262 21032 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:16:06.139807 21032 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:16:06.140637 21032 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:16:06.140730 21032 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:06.140794 21032 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:16:06.140810 21032 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:06.149745 21032 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.129:34799
I20260809 02:16:06.149788 21208 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.129:34799 every 8 connection(s)
I20260809 02:16:06.150599 21032 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
I20260809 02:16:06.154008 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 21032
I20260809 02:16:06.154134 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/minicluster-data/ts-0/wal/instance
I20260809 02:16:06.154445 15662 external_mini_cluster.cc:1428] Setting key f92da5d778e62ba5c9f96ad7cc84b8e1
Aug 09 02:16:06 dist-test-slave-s11t krb5kdc[20863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241766, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.129@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.75.130@KRBTEST.COM; defaulting to no policy
I20260809 02:16:06.165580 20998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:06.152328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.129:47151 (local address 127.15.75.190:44431)
0809 02:16:06.152464 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:06.152468 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:06.153305 (+ 837us) server_negotiation.cc:407] Connection header received
0809 02:16:06.154766 (+ 1461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:06.154769 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:06.154843 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:06.154919 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:06.156320 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:06.157182 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:06.158719 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:06.158968 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:06.159052 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:06.161898 (+ 2846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:06.161901 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:06.161903 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:06.161949 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:06.163467 (+ 1518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:06.163991 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:06.164000 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:06.164001 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:06.164079 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:06.164431 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:06.164435 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:06.164437 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:06.164618 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:06.164755 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:06.165278 (+ 523us) server_negotiation.cc:299] Negotiation successful
0809 02:16:06.165405 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 02:16:06.166422 21211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:06.152676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:44431 (local address 127.15.75.129:47151)
0809 02:16:06.153157 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:06.153198 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:06.154560 (+ 1362us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:06.155122 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:06.155129 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:06.155607 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:06.156171 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:06.156186 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:06.157324 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:06.157327 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:06.158576 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:06.158593 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:06.159139 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:06.159144 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:06.159381 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:06.159982 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:06.160007 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:06.161726 (+ 1719us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:16:06.163597 (+ 1871us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:06.163603 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:06.163616 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:06.163877 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:06.164175 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:06.164178 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:06.164180 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:06.164329 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:06.164755 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:16:06.164762 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:16:06.165046 (+ 284us) client_negotiation.cc:769] Sending connection context
0809 02:16:06.165236 (+ 190us) client_negotiation.cc:240] Negotiation successful
0809 02:16:06.165546 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":100,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.130@KRBTEST.COM" created.
I20260809 02:16:06.167595 21209 heartbeater.cc:347] Connected to a master server at 127.15.75.190:44431
I20260809 02:16:06.167850 21209 heartbeater.cc:464] Registering TS with master...
I20260809 02:16:06.168452 21209 heartbeater.cc:511] Master 127.15.75.190:44431 requested a full tablet report, sending...
I20260809 02:16:06.170269 20936 ts_manager.cc:194] Registered new tserver with Master: 4578b236c9cb4f4eba1b91e0372b3b75 (127.15.75.129:34799)
I20260809 02:16:06.171988 20936 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.75.129@KRBTEST.COM'} at 127.15.75.129:47151
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab.
Entry for principal kudu/127.15.75.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab.
I20260809 02:16:06.183034 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.130:0
--local_ip_for_outbound_sockets=127.15.75.130
--webserver_interface=127.15.75.130
--webserver_port=0
--tserver_master_addrs=127.15.75.190:44431
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab
--principal=kudu/127.15.75.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:41087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:16:06.297055 21214 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:16:06.297286 21214 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:16:06.297359 21214 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:16:06.301440 21214 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:16:06.301489 21214 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:16:06.301568 21214 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.130
I20260809 02:16:06.306797 21214 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:41087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab
--principal=kudu/127.15.75.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.75.130
--webserver_port=0
--tserver_master_addrs=127.15.75.190:44431
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21214
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.130
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:16:06.307788 21214 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:16:06.308841 21214 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:16:06.318470 21220 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:06.318475 21224 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:06.318588 21222 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:06.318609 21214 server_base.cc:1034] running on GCE node
I20260809 02:16:06.319125 21214 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:16:06.319739 21214 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:16:06.320933 21214 hybrid_clock.cc:648] HybridClock initialized: now 1786241766320926 us; error 35 us; skew 500 ppm
Aug 09 02:16:06 dist-test-slave-s11t krb5kdc[20863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241766, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:16:06.323721 21214 init.cc:377] Logged in from keytab as kudu/127.15.75.130@KRBTEST.COM (short username kudu)
I20260809 02:16:06.324508 21214 webserver.cc:460] Webserver started at http://127.15.75.130:35571/ using document root <none> and password file <none>
I20260809 02:16:06.325085 21214 fs_manager.cc:359] Metadata directory not provided
I20260809 02:16:06.325146 21214 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:16:06.325316 21214 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:16:06.327209 21214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/minicluster-data/ts-1/data/instance:
uuid: "6deb195262294c7190b28fdb78599fd2"
format_stamp: "Formatted at 2026-08-09 02:16:06 on dist-test-slave-s11t"
server_key: "550eb4f9367850d2d165169363970d76"
server_key_iv: "74b3d95758641b4488cb45626f542f4b"
server_key_version: "encryptionkey@0"
I20260809 02:16:06.327718 21214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/minicluster-data/ts-1/wal/instance:
uuid: "6deb195262294c7190b28fdb78599fd2"
format_stamp: "Formatted at 2026-08-09 02:16:06 on dist-test-slave-s11t"
server_key: "550eb4f9367850d2d165169363970d76"
server_key_iv: "74b3d95758641b4488cb45626f542f4b"
server_key_version: "encryptionkey@0"
I20260809 02:16:06.331382 21214 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260809 02:16:06.333751 21231 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:06.334885 21214 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 02:16:06.334976 21214 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/minicluster-data/ts-1/wal
uuid: "6deb195262294c7190b28fdb78599fd2"
format_stamp: "Formatted at 2026-08-09 02:16:06 on dist-test-slave-s11t"
server_key: "550eb4f9367850d2d165169363970d76"
server_key_iv: "74b3d95758641b4488cb45626f542f4b"
server_key_version: "encryptionkey@0"
I20260809 02:16:06.335074 21214 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:16:06.353567 21214 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:16:06.356604 21214 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:16:06.356815 21214 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:16:06.357311 21214 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:16:06.358050 21214 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:16:06.358096 21214 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:06.358132 21214 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:16:06.358153 21214 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:06.366859 21214 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.130:38445
I20260809 02:16:06.366917 21397 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.130:38445 every 8 connection(s)
I20260809 02:16:06.367743 21214 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
I20260809 02:16:06.369570 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 21214
I20260809 02:16:06.369699 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/minicluster-data/ts-1/wal/instance
I20260809 02:16:06.369952 15662 external_mini_cluster.cc:1428] Setting key 7f249ed31c527af8fb4f3cb949bd275c
Aug 09 02:16:06 dist-test-slave-s11t krb5kdc[20863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241766, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.130@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.75.131@KRBTEST.COM; defaulting to no policy
I20260809 02:16:06.381565 20998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:06.369539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.130:56409 (local address 127.15.75.190:44431)
0809 02:16:06.369698 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:06.369703 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:06.370480 (+ 777us) server_negotiation.cc:407] Connection header received
0809 02:16:06.371924 (+ 1444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:06.371927 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:06.371979 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:06.372064 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:06.373257 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:06.373792 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:06.374952 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:06.375182 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:06.375295 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:06.377954 (+ 2659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:06.377958 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:06.377960 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:06.377981 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:06.379427 (+ 1446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:06.379943 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:06.379947 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:06.379949 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:06.380032 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:06.380374 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:06.380379 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:06.380381 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:06.380569 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:06.380682 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:06.381259 (+ 577us) server_negotiation.cc:299] Negotiation successful
0809 02:16:06.381392 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 02:16:06.382380 21401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:06.369881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:44431 (local address 127.15.75.130:56409)
0809 02:16:06.370337 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:06.370387 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:06.371723 (+ 1336us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:06.372246 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:06.372252 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:06.372583 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:06.373116 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:06.373129 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:06.373935 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:06.373938 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:06.374785 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:06.374794 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:06.375333 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:06.375337 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:06.375502 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:06.376102 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:06.376127 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:06.377781 (+ 1654us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:16:06.379562 (+ 1781us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:06.379568 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:06.379582 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:06.379834 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:06.380134 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:06.380137 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:06.380139 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:06.380276 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:06.380707 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:16:06.380715 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:16:06.381026 (+ 311us) client_negotiation.cc:769] Sending connection context
0809 02:16:06.381215 (+ 189us) client_negotiation.cc:240] Negotiation successful
0809 02:16:06.381531 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":137,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.131@KRBTEST.COM" created.
I20260809 02:16:06.383550 21399 heartbeater.cc:347] Connected to a master server at 127.15.75.190:44431
I20260809 02:16:06.383805 21399 heartbeater.cc:464] Registering TS with master...
I20260809 02:16:06.384406 21399 heartbeater.cc:511] Master 127.15.75.190:44431 requested a full tablet report, sending...
I20260809 02:16:06.385571 20936 ts_manager.cc:194] Registered new tserver with Master: 6deb195262294c7190b28fdb78599fd2 (127.15.75.130:38445)
I20260809 02:16:06.386466 20936 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.75.130@KRBTEST.COM'} at 127.15.75.130:56409
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab.
Entry for principal kudu/127.15.75.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab.
I20260809 02:16:06.398010 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.131:0
--local_ip_for_outbound_sockets=127.15.75.131
--webserver_interface=127.15.75.131
--webserver_port=0
--tserver_master_addrs=127.15.75.190:44431
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab
--principal=kudu/127.15.75.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:41087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:16:06.509264 21404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:16:06.509514 21404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:16:06.509555 21404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:16:06.513540 21404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:16:06.513592 21404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:16:06.513649 21404 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.131
I20260809 02:16:06.518754 21404 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:41087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab
--principal=kudu/127.15.75.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.75.131
--webserver_port=0
--tserver_master_addrs=127.15.75.190:44431
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21404
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.131
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:16:06.519646 21404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:16:06.520658 21404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:16:06.530478 21415 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:06.530560 21404 server_base.cc:1034] running on GCE node
W20260809 02:16:06.530470 21420 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:06.530470 21417 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:06.531175 21404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:16:06.531791 21404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:16:06.532985 21404 hybrid_clock.cc:648] HybridClock initialized: now 1786241766532962 us; error 26 us; skew 500 ppm
Aug 09 02:16:06 dist-test-slave-s11t krb5kdc[20863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241766, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:16:06.535727 21404 init.cc:377] Logged in from keytab as kudu/127.15.75.131@KRBTEST.COM (short username kudu)
I20260809 02:16:06.536566 21404 webserver.cc:460] Webserver started at http://127.15.75.131:42635/ using document root <none> and password file <none>
I20260809 02:16:06.537146 21404 fs_manager.cc:359] Metadata directory not provided
I20260809 02:16:06.537209 21404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:16:06.537405 21404 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:16:06.539526 21404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/minicluster-data/ts-2/data/instance:
uuid: "565e36dc804f4a8d8511ce763793573d"
format_stamp: "Formatted at 2026-08-09 02:16:06 on dist-test-slave-s11t"
server_key: "b17a04cafab530fa3a9cb5508a76f185"
server_key_iv: "08956ae0160c534d1c455cbe2bfd06a0"
server_key_version: "encryptionkey@0"
I20260809 02:16:06.540120 21404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/minicluster-data/ts-2/wal/instance:
uuid: "565e36dc804f4a8d8511ce763793573d"
format_stamp: "Formatted at 2026-08-09 02:16:06 on dist-test-slave-s11t"
server_key: "b17a04cafab530fa3a9cb5508a76f185"
server_key_iv: "08956ae0160c534d1c455cbe2bfd06a0"
server_key_version: "encryptionkey@0"
I20260809 02:16:06.543910 21404 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 02:16:06.546279 21429 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:06.547459 21404 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 02:16:06.547554 21404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/minicluster-data/ts-2/wal
uuid: "565e36dc804f4a8d8511ce763793573d"
format_stamp: "Formatted at 2026-08-09 02:16:06 on dist-test-slave-s11t"
server_key: "b17a04cafab530fa3a9cb5508a76f185"
server_key_iv: "08956ae0160c534d1c455cbe2bfd06a0"
server_key_version: "encryptionkey@0"
I20260809 02:16:06.547638 21404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:16:06.561132 21404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:16:06.564339 21404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:16:06.564499 21404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:16:06.565132 21404 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:16:06.565944 21404 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:16:06.565992 21404 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:06.566031 21404 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:16:06.566053 21404 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:06.575521 21404 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.131:35847
I20260809 02:16:06.575556 21612 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.131:35847 every 8 connection(s)
I20260809 02:16:06.576432 21404 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
I20260809 02:16:06.584084 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 21404
I20260809 02:16:06.584250 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241741788589-15662-0/minicluster-data/ts-2/wal/instance
I20260809 02:16:06.584594 15662 external_mini_cluster.cc:1428] Setting key 9b502ee0d09f1ad010b69f7aa05cdbaf
Aug 09 02:16:06 dist-test-slave-s11t krb5kdc[20863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241766, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.131@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:16:06.590283 20998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:06.578314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.131:54949 (local address 127.15.75.190:44431)
0809 02:16:06.578476 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:06.578481 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:06.579398 (+ 917us) server_negotiation.cc:407] Connection header received
0809 02:16:06.580822 (+ 1424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:06.580826 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:06.580874 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:06.580950 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:06.582057 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:06.582594 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:06.583709 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:06.583940 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:06.584018 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:06.587194 (+ 3176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:06.587198 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:06.587201 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:06.587233 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:06.588434 (+ 1201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:06.588973 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:06.588976 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:06.588977 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:06.589051 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:06.589330 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:06.589332 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:06.589333 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:06.589496 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:06.589566 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:06.590063 (+ 497us) server_negotiation.cc:299] Negotiation successful
0809 02:16:06.590166 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 02:16:06.591049 21620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:06.578727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:44431 (local address 127.15.75.131:54949)
0809 02:16:06.579240 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:06.579283 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:06.580567 (+ 1284us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:06.581111 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:06.581118 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:06.581471 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:06.581935 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:06.581947 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:06.582716 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:06.582720 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:06.583603 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:06.583612 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:06.584116 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:06.584121 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:06.584339 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:06.585079 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:06.585095 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:06.587021 (+ 1926us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:16:06.588536 (+ 1515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:06.588541 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:06.588552 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:06.588887 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:06.589131 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:06.589133 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:06.589134 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:06.589269 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:06.589579 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:16:06.589584 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:16:06.589811 (+ 227us) client_negotiation.cc:769] Sending connection context
0809 02:16:06.589957 (+ 146us) client_negotiation.cc:240] Negotiation successful
0809 02:16:06.590190 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":121,"threads_started":1}
I20260809 02:16:06.592144 21613 heartbeater.cc:347] Connected to a master server at 127.15.75.190:44431
I20260809 02:16:06.592351 21613 heartbeater.cc:464] Registering TS with master...
I20260809 02:16:06.592851 21613 heartbeater.cc:511] Master 127.15.75.190:44431 requested a full tablet report, sending...
I20260809 02:16:06.593834 20936 ts_manager.cc:194] Registered new tserver with Master: 565e36dc804f4a8d8511ce763793573d (127.15.75.131:35847)
I20260809 02:16:06.594403 20936 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.75.131@KRBTEST.COM'} at 127.15.75.131:54949
I20260809 02:16:06.598431 15662 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 02:16:06 dist-test-slave-s11t krb5kdc[20863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241766, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 02:16:06 dist-test-slave-s11t krb5kdc[20863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241766, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.75.129@KRBTEST.COM
I20260809 02:16:06.625942 21631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:06.616512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53738 (local address 127.15.75.129:34799)
0809 02:16:06.616865 (+ 353us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:06.616880 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:06.616906 (+ 26us) server_negotiation.cc:407] Connection header received
0809 02:16:06.616958 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:06.616962 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:06.617146 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:06.617266 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:06.617924 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:06.618837 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:06.619620 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:06.619973 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:06.620053 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:06.622600 (+ 2547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:06.622609 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:06.622621 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:06.622644 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:06.624269 (+ 1625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:06.624633 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:06.624638 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:06.624639 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:06.624773 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:06.625014 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:06.625016 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:06.625017 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:06.625245 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:06.625410 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:06.625561 (+ 151us) server_negotiation.cc:299] Negotiation successful
0809 02:16:06.625728 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":87,"threads_started":1}
Aug 09 02:16:06 dist-test-slave-s11t krb5kdc[20863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241766, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:16:06.636489 20998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:06.628631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53032 (local address 127.15.75.190:44431)
0809 02:16:06.628813 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:06.628819 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:06.628836 (+ 17us) server_negotiation.cc:407] Connection header received
0809 02:16:06.628992 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:06.628995 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:06.629044 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:06.629130 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:06.629696 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:06.630202 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:06.630964 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:06.631190 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:06.631368 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:06.633638 (+ 2270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:06.633642 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:06.633644 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:06.633675 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:06.634835 (+ 1160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:06.635242 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:06.635247 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:06.635249 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:06.635329 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:06.635608 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:06.635612 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:06.635614 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:06.635828 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:06.635974 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:06.636209 (+ 235us) server_negotiation.cc:299] Negotiation successful
0809 02:16:06.636325 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
W20260809 02:16:06.636952 20936 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53032
I20260809 02:16:06.637573 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 21032
I20260809 02:16:06.653846 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 21214
I20260809 02:16:06.662863 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 21404
I20260809 02:16:06.672416 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 20883
2026-08-09T02:16:06Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2358 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:16:06 dist-test-slave-s11t krb5kdc[21640](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:16:06 dist-test-slave-s11t krb5kdc[21640](info): set up 2 sockets
Aug 09 02:16:06 dist-test-slave-s11t krb5kdc[21640](info): commencing operation
krb5kdc: starting...
W20260809 02:16:07.901497 15662 mini_kdc.cc:121] Time spent starting KDC: real 1.203s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Aug 09 02:16:07 dist-test-slave-s11t krb5kdc[21640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241767, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:16:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:16:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.75.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab.
Entry for principal kudu/127.15.75.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab.
I20260809 02:16:08.025159 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:45937
--webserver_interface=127.15.75.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab
--principal=kudu/127.15.75.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:43565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.75.190:45937
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:16:08.137841 21656 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:16:08.138069 21656 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:16:08.138108 21656 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:16:08.141932 21656 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:16:08.141974 21656 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:16:08.141993 21656 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:16:08.142009 21656 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:16:08.142024 21656 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:16:08.146956 21656 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:43565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.75.190:45937
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab
--principal=kudu/127.15.75.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:45937
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.75.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21656
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:16:08.147812 21656 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:16:08.148734 21656 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:16:08.156802 21665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:08.156870 21656 server_base.cc:1034] running on GCE node
W20260809 02:16:08.156777 21668 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:08.156773 21664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:08.157706 21656 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:16:08.158731 21656 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:16:08.159919 21656 hybrid_clock.cc:648] HybridClock initialized: now 1786241768159905 us; error 32 us; skew 500 ppm
Aug 09 02:16:08 dist-test-slave-s11t krb5kdc[21640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241768, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:16:08.162576 21656 init.cc:377] Logged in from keytab as kudu/127.15.75.190@KRBTEST.COM (short username kudu)
I20260809 02:16:08.163316 21656 fs_manager.cc:359] Metadata directory not provided
I20260809 02:16:08.163420 21656 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:16:08.163652 21656 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:16:08.165704 21656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/minicluster-data/master-0/data/instance:
uuid: "eeeadf730e854490a92ad5687b52172c"
format_stamp: "Formatted at 2026-08-09 02:16:08 on dist-test-slave-s11t"
server_key: "11e3d3a85d90928a991d594971ccefde"
server_key_iv: "c464ee3f8698b4bf8b15ad210880eb85"
server_key_version: "encryptionkey@0"
I20260809 02:16:08.166262 21656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/minicluster-data/master-0/wal/instance:
uuid: "eeeadf730e854490a92ad5687b52172c"
format_stamp: "Formatted at 2026-08-09 02:16:08 on dist-test-slave-s11t"
server_key: "11e3d3a85d90928a991d594971ccefde"
server_key_iv: "c464ee3f8698b4bf8b15ad210880eb85"
server_key_version: "encryptionkey@0"
I20260809 02:16:08.169842 21656 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 02:16:08.172091 21678 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:08.173301 21656 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 02:16:08.173447 21656 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/minicluster-data/master-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/minicluster-data/master-0/wal
uuid: "eeeadf730e854490a92ad5687b52172c"
format_stamp: "Formatted at 2026-08-09 02:16:08 on dist-test-slave-s11t"
server_key: "11e3d3a85d90928a991d594971ccefde"
server_key_iv: "c464ee3f8698b4bf8b15ad210880eb85"
server_key_version: "encryptionkey@0"
I20260809 02:16:08.173556 21656 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:16:08.192925 21656 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:16:08.196051 21656 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:16:08.196267 21656 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:16:08.203229 21656 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.190:45937
I20260809 02:16:08.203498 21760 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.190:45937 every 8 connection(s)
I20260809 02:16:08.203893 21656 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
I20260809 02:16:08.206584 21761 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:16:08.210925 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 21656
I20260809 02:16:08.211071 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/minicluster-data/master-0/wal/instance
I20260809 02:16:08.211356 15662 external_mini_cluster.cc:1428] Setting key 3bc9f98277bab8a0b33773635be6c5f4
I20260809 02:16:08.212862 21761 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eeeadf730e854490a92ad5687b52172c: Bootstrap starting.
I20260809 02:16:08.215687 21761 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eeeadf730e854490a92ad5687b52172c: Neither blocks nor log segments found. Creating new log.
I20260809 02:16:08.216574 21761 log.cc:824] T 00000000000000000000000000000000 P eeeadf730e854490a92ad5687b52172c: Log is configured to *not* fsync() on all Append() calls
I20260809 02:16:08.218636 21761 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eeeadf730e854490a92ad5687b52172c: No bootstrap required, opened a new log
Aug 09 02:16:08 dist-test-slave-s11t krb5kdc[21640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241767, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:16:08.222972 21761 raft_consensus.cc:348] T 00000000000000000000000000000000 P eeeadf730e854490a92ad5687b52172c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eeeadf730e854490a92ad5687b52172c" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 45937 } }
I20260809 02:16:08.223174 21761 raft_consensus.cc:374] T 00000000000000000000000000000000 P eeeadf730e854490a92ad5687b52172c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:16:08.223233 21761 raft_consensus.cc:729] T 00000000000000000000000000000000 P eeeadf730e854490a92ad5687b52172c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eeeadf730e854490a92ad5687b52172c, State: Initialized, Role: FOLLOWER
I20260809 02:16:08.223723 21761 consensus_queue.cc:260] T 00000000000000000000000000000000 P eeeadf730e854490a92ad5687b52172c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eeeadf730e854490a92ad5687b52172c" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 45937 } }
I20260809 02:16:08.223928 21761 raft_consensus.cc:388] T 00000000000000000000000000000000 P eeeadf730e854490a92ad5687b52172c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:16:08.224001 21761 raft_consensus.cc:482] T 00000000000000000000000000000000 P eeeadf730e854490a92ad5687b52172c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:16:08.224092 21761 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eeeadf730e854490a92ad5687b52172c [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:16:08.225013 21761 raft_consensus.cc:504] T 00000000000000000000000000000000 P eeeadf730e854490a92ad5687b52172c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eeeadf730e854490a92ad5687b52172c" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 45937 } }
I20260809 02:16:08.225349 21761 leader_election.cc:302] T 00000000000000000000000000000000 P eeeadf730e854490a92ad5687b52172c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eeeadf730e854490a92ad5687b52172c; no voters:
I20260809 02:16:08.225730 21761 leader_election.cc:288] T 00000000000000000000000000000000 P eeeadf730e854490a92ad5687b52172c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:16:08.225800 21768 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eeeadf730e854490a92ad5687b52172c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:16:08.226035 21768 raft_consensus.cc:686] T 00000000000000000000000000000000 P eeeadf730e854490a92ad5687b52172c [term 1 LEADER]: Becoming Leader. State: Replica: eeeadf730e854490a92ad5687b52172c, State: Running, Role: LEADER
I20260809 02:16:08.226394 21768 consensus_queue.cc:237] T 00000000000000000000000000000000 P eeeadf730e854490a92ad5687b52172c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eeeadf730e854490a92ad5687b52172c" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 45937 } }
I20260809 02:16:08.226608 21764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:08.212730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58802 (local address 127.15.75.190:45937)
0809 02:16:08.213194 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:08.213212 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:08.213255 (+ 43us) server_negotiation.cc:407] Connection header received
0809 02:16:08.214102 (+ 847us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:08.214126 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:08.214548 (+ 422us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:08.214897 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:08.215597 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:08.217114 (+ 1517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:08.218027 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:08.218695 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:08.218901 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:08.221609 (+ 2708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:08.221616 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:08.221632 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:08.221658 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:08.223380 (+ 1722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:08.223852 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:08.223873 (+ 21us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:08.223881 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:08.224048 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:08.224348 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:08.224351 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:08.224353 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:08.224801 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:08.225037 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:08.225278 (+ 241us) server_negotiation.cc:299] Negotiation successful
0809 02:16:08.225575 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":137,"threads_started":1}
I20260809 02:16:08.227001 21761 sys_catalog.cc:564] T 00000000000000000000000000000000 P eeeadf730e854490a92ad5687b52172c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:16:08.227609 21768 sys_catalog.cc:455] T 00000000000000000000000000000000 P eeeadf730e854490a92ad5687b52172c [sys.catalog]: SysCatalogTable state changed. Reason: New leader eeeadf730e854490a92ad5687b52172c. Latest consensus state: current_term: 1 leader_uuid: "eeeadf730e854490a92ad5687b52172c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eeeadf730e854490a92ad5687b52172c" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 45937 } } }
I20260809 02:16:08.227820 21768 sys_catalog.cc:458] T 00000000000000000000000000000000 P eeeadf730e854490a92ad5687b52172c [sys.catalog]: This master's current role is: LEADER
I20260809 02:16:08.228042 21770 sys_catalog.cc:455] T 00000000000000000000000000000000 P eeeadf730e854490a92ad5687b52172c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eeeadf730e854490a92ad5687b52172c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eeeadf730e854490a92ad5687b52172c" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 45937 } } }
I20260809 02:16:08.228147 21770 sys_catalog.cc:458] T 00000000000000000000000000000000 P eeeadf730e854490a92ad5687b52172c [sys.catalog]: This master's current role is: LEADER
W20260809 02:16:08.231781 21789 catalog_manager.cc:1558] T 00000000000000000000000000000000 P eeeadf730e854490a92ad5687b52172c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:16:08.231870 21789 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:16:08.231976 21781 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:16:08.232986 21781 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:16:08.238193 21781 catalog_manager.cc:1347] Generated new cluster ID: 66269301fe214dc4879a8e7863cb2eb7
I20260809 02:16:08.238256 21781 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:16:08.266602 21781 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:16:08.267582 21781 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:16:08.279868 21781 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eeeadf730e854490a92ad5687b52172c: Generated new TSK 0
I20260809 02:16:08.280560 21781 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.75.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab.
Entry for principal kudu/127.15.75.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab.
I20260809 02:16:08.309434 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.129:0
--local_ip_for_outbound_sockets=127.15.75.129
--webserver_interface=127.15.75.129
--webserver_port=0
--tserver_master_addrs=127.15.75.190:45937
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab
--principal=kudu/127.15.75.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:43565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:16:08.420219 21805 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:16:08.420466 21805 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:16:08.420532 21805 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:16:08.424402 21805 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:16:08.424481 21805 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:16:08.424578 21805 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.129
I20260809 02:16:08.429879 21805 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:43565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab
--principal=kudu/127.15.75.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.75.129
--webserver_port=0
--tserver_master_addrs=127.15.75.190:45937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21805
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.129
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:16:08.430806 21805 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:16:08.431726 21805 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:16:08.440958 21818 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:08.441004 21811 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:08.440994 21813 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:08.441100 21805 server_base.cc:1034] running on GCE node
I20260809 02:16:08.441497 21805 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:16:08.442078 21805 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:16:08.443238 21805 hybrid_clock.cc:648] HybridClock initialized: now 1786241768443225 us; error 25 us; skew 500 ppm
Aug 09 02:16:08 dist-test-slave-s11t krb5kdc[21640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241768, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:16:08.446089 21805 init.cc:377] Logged in from keytab as kudu/127.15.75.129@KRBTEST.COM (short username kudu)
I20260809 02:16:08.446789 21805 fs_manager.cc:359] Metadata directory not provided
I20260809 02:16:08.446882 21805 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:16:08.447104 21805 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:16:08.449131 21805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/minicluster-data/ts-0/data/instance:
uuid: "74ba4103613145e5b727b8efe7fa4d85"
format_stamp: "Formatted at 2026-08-09 02:16:08 on dist-test-slave-s11t"
server_key: "46445b1e840dfb5c16f6f887ebf778c8"
server_key_iv: "0cf2651a6ab28e74e465e3097ff637c9"
server_key_version: "encryptionkey@0"
I20260809 02:16:08.449684 21805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/minicluster-data/ts-0/wal/instance:
uuid: "74ba4103613145e5b727b8efe7fa4d85"
format_stamp: "Formatted at 2026-08-09 02:16:08 on dist-test-slave-s11t"
server_key: "46445b1e840dfb5c16f6f887ebf778c8"
server_key_iv: "0cf2651a6ab28e74e465e3097ff637c9"
server_key_version: "encryptionkey@0"
I20260809 02:16:08.453190 21805 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.002s
I20260809 02:16:08.455466 21825 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:08.456651 21805 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 02:16:08.456849 21805 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/minicluster-data/ts-0/wal
uuid: "74ba4103613145e5b727b8efe7fa4d85"
format_stamp: "Formatted at 2026-08-09 02:16:08 on dist-test-slave-s11t"
server_key: "46445b1e840dfb5c16f6f887ebf778c8"
server_key_iv: "0cf2651a6ab28e74e465e3097ff637c9"
server_key_version: "encryptionkey@0"
I20260809 02:16:08.456962 21805 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:16:08.478044 21805 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:16:08.481383 21805 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:16:08.481570 21805 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:16:08.482168 21805 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:16:08.483007 21805 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:16:08.483075 21805 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:08.483144 21805 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:16:08.483189 21805 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:08.492190 21805 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.129:35263
I20260809 02:16:08.492214 21998 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.129:35263 every 8 connection(s)
I20260809 02:16:08.492843 21805 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
I20260809 02:16:08.495723 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 21805
I20260809 02:16:08.495855 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/minicluster-data/ts-0/wal/instance
I20260809 02:16:08.496138 15662 external_mini_cluster.cc:1428] Setting key 6c6e7134ae27d1763cdcd2adc1dd52e2
Aug 09 02:16:08 dist-test-slave-s11t krb5kdc[21640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241768, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.129@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.75.130@KRBTEST.COM; defaulting to no policy
I20260809 02:16:08.506850 21764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:08.494550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.129:38405 (local address 127.15.75.190:45937)
0809 02:16:08.494680 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:08.494683 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:08.495512 (+ 829us) server_negotiation.cc:407] Connection header received
0809 02:16:08.497073 (+ 1561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:08.497075 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:08.497122 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:08.497205 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:08.498512 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:08.499361 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:08.500607 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:08.500930 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:08.501036 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:08.503360 (+ 2324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:08.503364 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:08.503366 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:08.503392 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:08.504920 (+ 1528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:08.505396 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:08.505402 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:08.505406 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:08.505485 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:08.505807 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:08.505812 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:08.505814 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:08.505990 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:08.506095 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:08.506555 (+ 460us) server_negotiation.cc:299] Negotiation successful
0809 02:16:08.506675 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 02:16:08.507697 22001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:08.494941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:45937 (local address 127.15.75.129:38405)
0809 02:16:08.495370 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:08.495418 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:08.496825 (+ 1407us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:08.497420 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:08.497430 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:08.497897 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:08.498346 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:08.498359 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:08.499491 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:08.499497 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:08.500476 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:08.500485 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:08.501085 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:08.501089 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:08.501260 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:08.501818 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:08.501833 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:08.503208 (+ 1375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:16:08.505030 (+ 1822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:08.505036 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:08.505049 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:08.505296 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:08.505574 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:08.505578 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:08.505581 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:08.505714 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:08.506093 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:16:08.506100 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:16:08.506329 (+ 229us) client_negotiation.cc:769] Sending connection context
0809 02:16:08.506512 (+ 183us) client_negotiation.cc:240] Negotiation successful
0809 02:16:08.506808 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":117,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.130@KRBTEST.COM" created.
I20260809 02:16:08.509059 21999 heartbeater.cc:347] Connected to a master server at 127.15.75.190:45937
I20260809 02:16:08.509294 21999 heartbeater.cc:464] Registering TS with master...
I20260809 02:16:08.509717 21999 heartbeater.cc:511] Master 127.15.75.190:45937 requested a full tablet report, sending...
I20260809 02:16:08.511155 21702 ts_manager.cc:194] Registered new tserver with Master: 74ba4103613145e5b727b8efe7fa4d85 (127.15.75.129:35263)
I20260809 02:16:08.512261 21702 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.75.129@KRBTEST.COM'} at 127.15.75.129:38405
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab.
Entry for principal kudu/127.15.75.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab.
I20260809 02:16:08.523061 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.130:0
--local_ip_for_outbound_sockets=127.15.75.130
--webserver_interface=127.15.75.130
--webserver_port=0
--tserver_master_addrs=127.15.75.190:45937
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab
--principal=kudu/127.15.75.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:43565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:16:08.633342 22004 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:16:08.633678 22004 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:16:08.633764 22004 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:16:08.637740 22004 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:16:08.637795 22004 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:16:08.637894 22004 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.130
I20260809 02:16:08.642973 22004 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:43565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab
--principal=kudu/127.15.75.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.75.130
--webserver_port=0
--tserver_master_addrs=127.15.75.190:45937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22004
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.130
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:16:08.643950 22004 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:16:08.644964 22004 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:16:08.654101 22014 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:08.654147 22010 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:08.654098 22016 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:08.654352 22004 server_base.cc:1034] running on GCE node
I20260809 02:16:08.654808 22004 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:16:08.655414 22004 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:16:08.656570 22004 hybrid_clock.cc:648] HybridClock initialized: now 1786241768656555 us; error 31 us; skew 500 ppm
Aug 09 02:16:08 dist-test-slave-s11t krb5kdc[21640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241768, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:16:08.659327 22004 init.cc:377] Logged in from keytab as kudu/127.15.75.130@KRBTEST.COM (short username kudu)
I20260809 02:16:08.660085 22004 fs_manager.cc:359] Metadata directory not provided
I20260809 02:16:08.660177 22004 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:16:08.660386 22004 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:16:08.662467 22004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/minicluster-data/ts-1/data/instance:
uuid: "265db3c10c97462380b7418f3bb5c283"
format_stamp: "Formatted at 2026-08-09 02:16:08 on dist-test-slave-s11t"
server_key: "4457e199051509af8273aaeda5892a95"
server_key_iv: "2887482ae2143b66e87520152acd64eb"
server_key_version: "encryptionkey@0"
I20260809 02:16:08.663043 22004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/minicluster-data/ts-1/wal/instance:
uuid: "265db3c10c97462380b7418f3bb5c283"
format_stamp: "Formatted at 2026-08-09 02:16:08 on dist-test-slave-s11t"
server_key: "4457e199051509af8273aaeda5892a95"
server_key_iv: "2887482ae2143b66e87520152acd64eb"
server_key_version: "encryptionkey@0"
I20260809 02:16:08.666605 22004 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 02:16:08.668951 22024 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:08.670240 22004 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 02:16:08.670385 22004 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/minicluster-data/ts-1/wal
uuid: "265db3c10c97462380b7418f3bb5c283"
format_stamp: "Formatted at 2026-08-09 02:16:08 on dist-test-slave-s11t"
server_key: "4457e199051509af8273aaeda5892a95"
server_key_iv: "2887482ae2143b66e87520152acd64eb"
server_key_version: "encryptionkey@0"
I20260809 02:16:08.670500 22004 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:16:08.706037 22004 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:16:08.709834 22004 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:16:08.710058 22004 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:16:08.710709 22004 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:16:08.711611 22004 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:16:08.711663 22004 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:08.711750 22004 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:16:08.711790 22004 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:08.721473 22004 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.130:46489
I20260809 02:16:08.721501 22185 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.130:46489 every 8 connection(s)
I20260809 02:16:08.722097 22004 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
I20260809 02:16:08.729218 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 22004
I20260809 02:16:08.729327 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/minicluster-data/ts-1/wal/instance
I20260809 02:16:08.729641 15662 external_mini_cluster.cc:1428] Setting key 6e7dcbb32f3f2385a85980c78fa300bf
Aug 09 02:16:08 dist-test-slave-s11t krb5kdc[21640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241768, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.130@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:16:08.737047 21764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:08.723818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.130:47275 (local address 127.15.75.190:45937)
0809 02:16:08.723960 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:08.723965 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:08.724788 (+ 823us) server_negotiation.cc:407] Connection header received
0809 02:16:08.726229 (+ 1441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:08.726233 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:08.726309 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:08.726399 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:08.727707 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:08.728443 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:08.729912 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:08.730177 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:08.730303 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:08.733153 (+ 2850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:08.733163 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:08.733165 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:08.733189 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:08.734822 (+ 1633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:08.735384 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:08.735388 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:08.735389 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:08.735469 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:08.735850 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:08.735854 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:08.735856 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:08.736041 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:08.736156 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:08.736672 (+ 516us) server_negotiation.cc:299] Negotiation successful
0809 02:16:08.736863 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 02:16:08.737821 22188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:08.724160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:45937 (local address 127.15.75.130:47275)
0809 02:16:08.724592 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:08.724640 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:08.726008 (+ 1368us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:08.726582 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:08.726590 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:08.726967 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:08.727558 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:08.727576 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:08.728588 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:08.728595 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:08.729777 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:08.729788 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:08.730349 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:08.730354 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:08.730580 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:08.731199 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:08.731216 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:08.732982 (+ 1766us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:16:08.734954 (+ 1972us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:08.734961 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:08.734976 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:08.735253 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:08.735569 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:08.735574 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:08.735576 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:08.735726 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:08.736155 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:16:08.736162 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:16:08.736415 (+ 253us) client_negotiation.cc:769] Sending connection context
0809 02:16:08.736603 (+ 188us) client_negotiation.cc:240] Negotiation successful
0809 02:16:08.736964 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":107,"threads_started":1}
I20260809 02:16:08.738977 22186 heartbeater.cc:347] Connected to a master server at 127.15.75.190:45937
I20260809 02:16:08.739224 22186 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.15.75.131@KRBTEST.COM; defaulting to no policy
I20260809 02:16:08.739724 22186 heartbeater.cc:511] Master 127.15.75.190:45937 requested a full tablet report, sending...
I20260809 02:16:08.740796 21702 ts_manager.cc:194] Registered new tserver with Master: 265db3c10c97462380b7418f3bb5c283 (127.15.75.130:46489)
I20260809 02:16:08.741369 21702 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.75.130@KRBTEST.COM'} at 127.15.75.130:47275
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab.
Entry for principal kudu/127.15.75.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab.
I20260809 02:16:08.755419 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.131:0
--local_ip_for_outbound_sockets=127.15.75.131
--webserver_interface=127.15.75.131
--webserver_port=0
--tserver_master_addrs=127.15.75.190:45937
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab
--principal=kudu/127.15.75.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:43565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:16:08.865705 22194 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:16:08.865926 22194 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:16:08.865968 22194 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:16:08.869822 22194 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:16:08.869867 22194 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:16:08.869923 22194 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.131
I20260809 02:16:08.874990 22194 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:43565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab
--principal=kudu/127.15.75.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.75.131
--webserver_port=0
--tserver_master_addrs=127.15.75.190:45937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22194
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.131
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:16:08.876180 22194 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:16:08.877213 22194 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:16:08.886693 22206 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:08.886694 22199 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:08.886750 22200 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:08.886963 22194 server_base.cc:1034] running on GCE node
I20260809 02:16:08.887387 22194 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:16:08.887982 22194 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:16:08.889161 22194 hybrid_clock.cc:648] HybridClock initialized: now 1786241768889150 us; error 30 us; skew 500 ppm
Aug 09 02:16:08 dist-test-slave-s11t krb5kdc[21640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241768, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:16:08.891959 22194 init.cc:377] Logged in from keytab as kudu/127.15.75.131@KRBTEST.COM (short username kudu)
I20260809 02:16:08.892652 22194 fs_manager.cc:359] Metadata directory not provided
I20260809 02:16:08.892776 22194 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:16:08.893008 22194 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:16:08.895044 22194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/minicluster-data/ts-2/data/instance:
uuid: "17a8ebde02364146967673d58b1560f4"
format_stamp: "Formatted at 2026-08-09 02:16:08 on dist-test-slave-s11t"
server_key: "68bdd757b5dbbbe6c3675c9d2783aa6b"
server_key_iv: "1db6541baa6a8bf15abd2f416cd8a8d4"
server_key_version: "encryptionkey@0"
I20260809 02:16:08.895618 22194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/minicluster-data/ts-2/wal/instance:
uuid: "17a8ebde02364146967673d58b1560f4"
format_stamp: "Formatted at 2026-08-09 02:16:08 on dist-test-slave-s11t"
server_key: "68bdd757b5dbbbe6c3675c9d2783aa6b"
server_key_iv: "1db6541baa6a8bf15abd2f416cd8a8d4"
server_key_version: "encryptionkey@0"
I20260809 02:16:08.899174 22194 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260809 02:16:08.901582 22212 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:08.902720 22194 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 02:16:08.902855 22194 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/minicluster-data/ts-2/wal
uuid: "17a8ebde02364146967673d58b1560f4"
format_stamp: "Formatted at 2026-08-09 02:16:08 on dist-test-slave-s11t"
server_key: "68bdd757b5dbbbe6c3675c9d2783aa6b"
server_key_iv: "1db6541baa6a8bf15abd2f416cd8a8d4"
server_key_version: "encryptionkey@0"
I20260809 02:16:08.902959 22194 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:16:08.923187 22194 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:16:08.926376 22194 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:16:08.926570 22194 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:16:08.927217 22194 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:16:08.928129 22194 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:16:08.928196 22194 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:08.928267 22194 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:16:08.928310 22194 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:08.937131 22194 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.131:32893
I20260809 02:16:08.937153 22383 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.131:32893 every 8 connection(s)
I20260809 02:16:08.937741 22194 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
I20260809 02:16:08.941303 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 22194
I20260809 02:16:08.941435 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/minicluster-data/ts-2/wal/instance
I20260809 02:16:08.941718 15662 external_mini_cluster.cc:1428] Setting key 4297fd7d9ff191cce94d76b70da98041
Aug 09 02:16:08 dist-test-slave-s11t krb5kdc[21640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241768, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.131@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:16:08.951004 21764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:08.939347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.131:36907 (local address 127.15.75.190:45937)
0809 02:16:08.939495 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:08.939499 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:08.940302 (+ 803us) server_negotiation.cc:407] Connection header received
0809 02:16:08.941689 (+ 1387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:08.941692 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:08.941751 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:08.941851 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:08.943325 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:08.943961 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:08.945149 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:08.945357 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:08.945436 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:08.947718 (+ 2282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:08.947721 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:08.947724 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:08.947749 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:08.949084 (+ 1335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:08.949570 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:08.949574 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:08.949576 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:08.949656 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:08.949974 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:08.949979 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:08.949980 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:08.950154 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:08.950256 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:08.950705 (+ 449us) server_negotiation.cc:299] Negotiation successful
0809 02:16:08.950823 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 02:16:08.951668 22387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:08.939721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:45937 (local address 127.15.75.131:36907)
0809 02:16:08.940159 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:08.940200 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:08.941458 (+ 1258us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:08.942073 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:08.942080 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:08.942550 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:08.943188 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:08.943204 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:08.944073 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:08.944076 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:08.945045 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:08.945055 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:08.945478 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:08.945481 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:08.945609 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:08.946138 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:08.946154 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:08.947561 (+ 1407us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:16:08.949202 (+ 1641us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:08.949207 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:08.949220 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:08.949469 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:08.949745 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:08.949750 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:08.949751 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:08.949881 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:08.950257 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:16:08.950264 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:16:08.950489 (+ 225us) client_negotiation.cc:769] Sending connection context
0809 02:16:08.950665 (+ 176us) client_negotiation.cc:240] Negotiation successful
0809 02:16:08.950951 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":123,"threads_started":1}
I20260809 02:16:08.952646 22384 heartbeater.cc:347] Connected to a master server at 127.15.75.190:45937
I20260809 02:16:08.952950 22384 heartbeater.cc:464] Registering TS with master...
I20260809 02:16:08.953389 22384 heartbeater.cc:511] Master 127.15.75.190:45937 requested a full tablet report, sending...
I20260809 02:16:08.954483 21702 ts_manager.cc:194] Registered new tserver with Master: 17a8ebde02364146967673d58b1560f4 (127.15.75.131:32893)
I20260809 02:16:08.955116 21702 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.75.131@KRBTEST.COM'} at 127.15.75.131:36907
I20260809 02:16:08.955605 15662 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 02:16:08.964085 21764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:08.957555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58808 (local address 127.15.75.190:45937)
0809 02:16:08.957641 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:08.957644 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:08.957871 (+ 227us) server_negotiation.cc:407] Connection header received
0809 02:16:08.957951 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:08.957953 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:08.957988 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:08.958047 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:08.958653 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:08.959123 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:08.959867 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:08.960067 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:08.960144 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:08.961430 (+ 1286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:08.961434 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:08.961437 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:08.961463 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:08.962816 (+ 1353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:08.963167 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:08.963169 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:08.963170 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:08.963239 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:08.963467 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:08.963469 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:08.963470 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:08.963651 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:08.963748 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:08.963901 (+ 153us) server_negotiation.cc:299] Negotiation successful
0809 02:16:08.963982 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 02:16:08.966619 21702 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58808:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:16:08.969477 21702 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:16:08.983771 22402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:08.977805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.130:46489 (local address 127.0.0.1:46850)
0809 02:16:08.978738 (+ 933us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:08.978788 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:08.978926 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:08.979595 (+ 669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:08.979601 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:08.979623 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:16:08.979773 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:08.979781 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:08.981218 (+ 1437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:08.981221 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:08.982385 (+ 1164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:08.982396 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:08.983243 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:08.983247 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:08.983353 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:08.983389 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 02:16:08.983500 (+ 111us) client_negotiation.cc:240] Negotiation successful
0809 02:16:08.983615 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":812,"mutex_wait_us":5,"spinlock_wait_cycles":6272,"thread_start_us":115,"threads_started":1}
I20260809 02:16:08.983775 22405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:08.978488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46850 (local address 127.15.75.130:46489)
0809 02:16:08.978882 (+ 394us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:08.978896 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:08.978921 (+ 25us) server_negotiation.cc:407] Connection header received
0809 02:16:08.979232 (+ 311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:08.979238 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:08.979436 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:08.979601 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:16:08.979897 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:08.981084 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:08.982509 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:08.983112 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:08.983243 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:08.983399 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:08.983528 (+ 129us) server_negotiation.cc:299] Negotiation successful
0809 02:16:08.983615 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":123,"threads_started":1}
I20260809 02:16:08.984390 22400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:08.978044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.129:35263 (local address 127.0.0.1:40496)
0809 02:16:08.979178 (+ 1134us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:08.979192 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:08.979279 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:08.979938 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:08.979940 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:08.979955 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:16:08.980060 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:08.980067 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:08.981452 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:08.981457 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:08.982478 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:08.982491 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:08.983977 (+ 1486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:08.983985 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:08.984093 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:08.984117 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 02:16:08.984192 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 02:16:08.984258 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1072,"thread_start_us":81,"threads_started":1}
I20260809 02:16:08.984899 22404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:08.978472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40496 (local address 127.15.75.129:35263)
0809 02:16:08.979132 (+ 660us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:08.979139 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:08.979376 (+ 237us) server_negotiation.cc:407] Connection header received
0809 02:16:08.979449 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:08.979453 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:08.979631 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:08.979774 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:16:08.980167 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:08.981298 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:08.982605 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:08.983415 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:08.983537 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:08.983677 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:08.984613 (+ 936us) server_negotiation.cc:299] Negotiation successful
0809 02:16:08.984747 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":532,"thread_start_us":76,"threads_started":1}
I20260809 02:16:08.985169 22399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:08.977704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.131:32893 (local address 127.0.0.1:60038)
0809 02:16:08.978738 (+ 1034us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:08.978790 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:08.978926 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:08.979808 (+ 882us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:08.979810 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:08.979825 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:16:08.979960 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:08.979966 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:08.981047 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:08.981052 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:08.983449 (+ 2397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:08.983462 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:08.984741 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:08.984745 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:08.984850 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:08.984892 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 02:16:08.984964 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 02:16:08.985038 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":843,"mutex_wait_us":1148,"spinlock_wait_cycles":12288,"thread_start_us":81,"threads_started":1}
I20260809 02:16:08.985615 22401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:08.977841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60038 (local address 127.15.75.131:32893)
0809 02:16:08.979254 (+ 1413us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:08.979264 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:08.979293 (+ 29us) server_negotiation.cc:407] Connection header received
0809 02:16:08.979355 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:08.979360 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:08.979508 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:08.979652 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:16:08.980090 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:08.980918 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:08.983586 (+ 2668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:08.984400 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:08.984533 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:08.984657 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:08.985346 (+ 689us) server_negotiation.cc:299] Negotiation successful
0809 02:16:08.985455 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1323,"thread_start_us":187,"threads_started":1}
I20260809 02:16:08.986909 22093 tablet_service.cc:1467] Processing CreateTablet for tablet dd42e2d1eaf849af8a49a917074cbe6e (DEFAULT_TABLE table=test-table [id=4921fd1dc6be4a69894a9a7e238b8f8f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:16:08.987870 22093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd42e2d1eaf849af8a49a917074cbe6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:16:08.987840 22292 tablet_service.cc:1467] Processing CreateTablet for tablet dd42e2d1eaf849af8a49a917074cbe6e (DEFAULT_TABLE table=test-table [id=4921fd1dc6be4a69894a9a7e238b8f8f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:16:08.988823 22292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd42e2d1eaf849af8a49a917074cbe6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:16:08.991255 21891 tablet_service.cc:1467] Processing CreateTablet for tablet dd42e2d1eaf849af8a49a917074cbe6e (DEFAULT_TABLE table=test-table [id=4921fd1dc6be4a69894a9a7e238b8f8f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:16:08.992221 21891 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd42e2d1eaf849af8a49a917074cbe6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:16:08.994704 22406 tablet_bootstrap.cc:492] T dd42e2d1eaf849af8a49a917074cbe6e P 17a8ebde02364146967673d58b1560f4: Bootstrap starting.
I20260809 02:16:08.995903 22408 tablet_bootstrap.cc:492] T dd42e2d1eaf849af8a49a917074cbe6e P 265db3c10c97462380b7418f3bb5c283: Bootstrap starting.
I20260809 02:16:08.996816 22406 tablet_bootstrap.cc:654] T dd42e2d1eaf849af8a49a917074cbe6e P 17a8ebde02364146967673d58b1560f4: Neither blocks nor log segments found. Creating new log.
I20260809 02:16:08.997905 22406 log.cc:824] T dd42e2d1eaf849af8a49a917074cbe6e P 17a8ebde02364146967673d58b1560f4: Log is configured to *not* fsync() on all Append() calls
I20260809 02:16:08.998100 22408 tablet_bootstrap.cc:654] T dd42e2d1eaf849af8a49a917074cbe6e P 265db3c10c97462380b7418f3bb5c283: Neither blocks nor log segments found. Creating new log.
I20260809 02:16:08.999156 22412 tablet_bootstrap.cc:492] T dd42e2d1eaf849af8a49a917074cbe6e P 74ba4103613145e5b727b8efe7fa4d85: Bootstrap starting.
I20260809 02:16:08.999315 22408 log.cc:824] T dd42e2d1eaf849af8a49a917074cbe6e P 265db3c10c97462380b7418f3bb5c283: Log is configured to *not* fsync() on all Append() calls
I20260809 02:16:08.999881 22406 tablet_bootstrap.cc:492] T dd42e2d1eaf849af8a49a917074cbe6e P 17a8ebde02364146967673d58b1560f4: No bootstrap required, opened a new log
I20260809 02:16:09.000080 22406 ts_tablet_manager.cc:1397] T dd42e2d1eaf849af8a49a917074cbe6e P 17a8ebde02364146967673d58b1560f4: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260809 02:16:09.001149 22408 tablet_bootstrap.cc:492] T dd42e2d1eaf849af8a49a917074cbe6e P 265db3c10c97462380b7418f3bb5c283: No bootstrap required, opened a new log
I20260809 02:16:09.001323 22408 ts_tablet_manager.cc:1397] T dd42e2d1eaf849af8a49a917074cbe6e P 265db3c10c97462380b7418f3bb5c283: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260809 02:16:09.001422 22412 tablet_bootstrap.cc:654] T dd42e2d1eaf849af8a49a917074cbe6e P 74ba4103613145e5b727b8efe7fa4d85: Neither blocks nor log segments found. Creating new log.
I20260809 02:16:09.002574 22412 log.cc:824] T dd42e2d1eaf849af8a49a917074cbe6e P 74ba4103613145e5b727b8efe7fa4d85: Log is configured to *not* fsync() on all Append() calls
I20260809 02:16:09.004004 22406 raft_consensus.cc:348] T dd42e2d1eaf849af8a49a917074cbe6e P 17a8ebde02364146967673d58b1560f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17a8ebde02364146967673d58b1560f4" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 32893 } } peers { permanent_uuid: "265db3c10c97462380b7418f3bb5c283" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 46489 } } peers { permanent_uuid: "74ba4103613145e5b727b8efe7fa4d85" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 35263 } }
I20260809 02:16:09.004092 22408 raft_consensus.cc:348] T dd42e2d1eaf849af8a49a917074cbe6e P 265db3c10c97462380b7418f3bb5c283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17a8ebde02364146967673d58b1560f4" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 32893 } } peers { permanent_uuid: "265db3c10c97462380b7418f3bb5c283" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 46489 } } peers { permanent_uuid: "74ba4103613145e5b727b8efe7fa4d85" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 35263 } }
I20260809 02:16:09.004285 22406 raft_consensus.cc:374] T dd42e2d1eaf849af8a49a917074cbe6e P 17a8ebde02364146967673d58b1560f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:16:09.004317 22408 raft_consensus.cc:374] T dd42e2d1eaf849af8a49a917074cbe6e P 265db3c10c97462380b7418f3bb5c283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:16:09.004346 22406 raft_consensus.cc:729] T dd42e2d1eaf849af8a49a917074cbe6e P 17a8ebde02364146967673d58b1560f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17a8ebde02364146967673d58b1560f4, State: Initialized, Role: FOLLOWER
I20260809 02:16:09.004366 22408 raft_consensus.cc:729] T dd42e2d1eaf849af8a49a917074cbe6e P 265db3c10c97462380b7418f3bb5c283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 265db3c10c97462380b7418f3bb5c283, State: Initialized, Role: FOLLOWER
I20260809 02:16:09.004513 22412 tablet_bootstrap.cc:492] T dd42e2d1eaf849af8a49a917074cbe6e P 74ba4103613145e5b727b8efe7fa4d85: No bootstrap required, opened a new log
I20260809 02:16:09.004707 22412 ts_tablet_manager.cc:1397] T dd42e2d1eaf849af8a49a917074cbe6e P 74ba4103613145e5b727b8efe7fa4d85: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260809 02:16:09.004850 22408 consensus_queue.cc:260] T dd42e2d1eaf849af8a49a917074cbe6e P 265db3c10c97462380b7418f3bb5c283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17a8ebde02364146967673d58b1560f4" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 32893 } } peers { permanent_uuid: "265db3c10c97462380b7418f3bb5c283" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 46489 } } peers { permanent_uuid: "74ba4103613145e5b727b8efe7fa4d85" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 35263 } }
I20260809 02:16:09.004851 22406 consensus_queue.cc:260] T dd42e2d1eaf849af8a49a917074cbe6e P 17a8ebde02364146967673d58b1560f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17a8ebde02364146967673d58b1560f4" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 32893 } } peers { permanent_uuid: "265db3c10c97462380b7418f3bb5c283" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 46489 } } peers { permanent_uuid: "74ba4103613145e5b727b8efe7fa4d85" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 35263 } }
I20260809 02:16:09.005712 22186 heartbeater.cc:503] Master 127.15.75.190:45937 was elected leader, sending a full tablet report...
I20260809 02:16:09.005829 22406 ts_tablet_manager.cc:1428] T dd42e2d1eaf849af8a49a917074cbe6e P 17a8ebde02364146967673d58b1560f4: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 02:16:09.005877 22408 ts_tablet_manager.cc:1428] T dd42e2d1eaf849af8a49a917074cbe6e P 265db3c10c97462380b7418f3bb5c283: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260809 02:16:09.006917 22384 heartbeater.cc:503] Master 127.15.75.190:45937 was elected leader, sending a full tablet report...
I20260809 02:16:09.008229 22412 raft_consensus.cc:348] T dd42e2d1eaf849af8a49a917074cbe6e P 74ba4103613145e5b727b8efe7fa4d85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17a8ebde02364146967673d58b1560f4" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 32893 } } peers { permanent_uuid: "265db3c10c97462380b7418f3bb5c283" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 46489 } } peers { permanent_uuid: "74ba4103613145e5b727b8efe7fa4d85" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 35263 } }
I20260809 02:16:09.008520 22412 raft_consensus.cc:374] T dd42e2d1eaf849af8a49a917074cbe6e P 74ba4103613145e5b727b8efe7fa4d85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:16:09.008589 22412 raft_consensus.cc:729] T dd42e2d1eaf849af8a49a917074cbe6e P 74ba4103613145e5b727b8efe7fa4d85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74ba4103613145e5b727b8efe7fa4d85, State: Initialized, Role: FOLLOWER
I20260809 02:16:09.009020 22412 consensus_queue.cc:260] T dd42e2d1eaf849af8a49a917074cbe6e P 74ba4103613145e5b727b8efe7fa4d85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17a8ebde02364146967673d58b1560f4" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 32893 } } peers { permanent_uuid: "265db3c10c97462380b7418f3bb5c283" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 46489 } } peers { permanent_uuid: "74ba4103613145e5b727b8efe7fa4d85" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 35263 } }
I20260809 02:16:09.009734 21999 heartbeater.cc:503] Master 127.15.75.190:45937 was elected leader, sending a full tablet report...
I20260809 02:16:09.009958 22412 ts_tablet_manager.cc:1428] T dd42e2d1eaf849af8a49a917074cbe6e P 74ba4103613145e5b727b8efe7fa4d85: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
W20260809 02:16:09.188637 22386 tablet.cc:2369] T dd42e2d1eaf849af8a49a917074cbe6e P 17a8ebde02364146967673d58b1560f4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 02:16:09.223227 22187 tablet.cc:2369] T dd42e2d1eaf849af8a49a917074cbe6e P 265db3c10c97462380b7418f3bb5c283: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 02:16:09.244115 22000 tablet.cc:2369] T dd42e2d1eaf849af8a49a917074cbe6e P 74ba4103613145e5b727b8efe7fa4d85: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:16:09.307780 22419 raft_consensus.cc:482] T dd42e2d1eaf849af8a49a917074cbe6e P 17a8ebde02364146967673d58b1560f4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:16:09.308053 22419 raft_consensus.cc:504] T dd42e2d1eaf849af8a49a917074cbe6e P 17a8ebde02364146967673d58b1560f4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17a8ebde02364146967673d58b1560f4" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 32893 } } peers { permanent_uuid: "265db3c10c97462380b7418f3bb5c283" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 46489 } } peers { permanent_uuid: "74ba4103613145e5b727b8efe7fa4d85" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 35263 } }
I20260809 02:16:09.309777 22419 leader_election.cc:288] T dd42e2d1eaf849af8a49a917074cbe6e P 17a8ebde02364146967673d58b1560f4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 265db3c10c97462380b7418f3bb5c283 (127.15.75.130:46489), 74ba4103613145e5b727b8efe7fa4d85 (127.15.75.129:35263)
I20260809 02:16:09.313292 22405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:09.309688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.131:60035 (local address 127.15.75.130:46489)
0809 02:16:09.310056 (+ 368us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:09.310061 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:09.310216 (+ 155us) server_negotiation.cc:407] Connection header received
0809 02:16:09.310342 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:09.310345 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:09.310400 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:09.310487 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:16:09.310766 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:09.311286 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:09.312333 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:09.312742 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:09.312856 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:09.312921 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:09.313085 (+ 164us) server_negotiation.cc:299] Negotiation successful
0809 02:16:09.313169 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267}
I20260809 02:16:09.313294 22423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:09.309630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.130:46489 (local address 127.15.75.131:60035)
0809 02:16:09.310103 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:09.310116 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:09.310234 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:09.310518 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:09.310521 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:09.310539 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:16:09.310657 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:09.310664 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:09.311395 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:09.311398 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:09.312174 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:09.312182 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:09.312853 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:09.312857 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:09.312952 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:09.312982 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 02:16:09.313050 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 02:16:09.313105 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":412,"mutex_wait_us":55,"thread_start_us":122,"threads_started":1}
I20260809 02:16:09.313308 22387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:09.309628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.129:35263 (local address 127.15.75.131:52103)
0809 02:16:09.309830 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:09.309847 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:09.309957 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:09.310250 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:09.310253 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:09.310291 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:16:09.310437 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:09.310447 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:09.311286 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:09.311290 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:09.312165 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:09.312175 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:09.312909 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:09.312913 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:09.313024 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:09.313066 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 02:16:09.313144 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 02:16:09.313210 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111}
I20260809 02:16:09.313366 22404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:09.309757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.131:52103 (local address 127.15.75.129:35263)
0809 02:16:09.309902 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:09.309907 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:09.309926 (+ 19us) server_negotiation.cc:407] Connection header received
0809 02:16:09.310074 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:09.310078 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:09.310140 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:09.310226 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:16:09.310554 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:09.311172 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:09.312298 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:09.312763 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:09.312892 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:09.312945 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:09.313175 (+ 230us) server_negotiation.cc:299] Negotiation successful
0809 02:16:09.313241 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 02:16:09.313987 22120 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd42e2d1eaf849af8a49a917074cbe6e" candidate_uuid: "17a8ebde02364146967673d58b1560f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "265db3c10c97462380b7418f3bb5c283" is_pre_election: true
I20260809 02:16:09.313988 21922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd42e2d1eaf849af8a49a917074cbe6e" candidate_uuid: "17a8ebde02364146967673d58b1560f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74ba4103613145e5b727b8efe7fa4d85" is_pre_election: true
I20260809 02:16:09.314288 22120 raft_consensus.cc:2454] T dd42e2d1eaf849af8a49a917074cbe6e P 265db3c10c97462380b7418f3bb5c283 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17a8ebde02364146967673d58b1560f4 in term 0.
I20260809 02:16:09.314288 21922 raft_consensus.cc:2454] T dd42e2d1eaf849af8a49a917074cbe6e P 74ba4103613145e5b727b8efe7fa4d85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17a8ebde02364146967673d58b1560f4 in term 0.
I20260809 02:16:09.314747 22226 leader_election.cc:302] T dd42e2d1eaf849af8a49a917074cbe6e P 17a8ebde02364146967673d58b1560f4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17a8ebde02364146967673d58b1560f4, 265db3c10c97462380b7418f3bb5c283; no voters:
I20260809 02:16:09.315024 22419 raft_consensus.cc:2781] T dd42e2d1eaf849af8a49a917074cbe6e P 17a8ebde02364146967673d58b1560f4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:16:09.315110 22419 raft_consensus.cc:482] T dd42e2d1eaf849af8a49a917074cbe6e P 17a8ebde02364146967673d58b1560f4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:16:09.315155 22419 raft_consensus.cc:3037] T dd42e2d1eaf849af8a49a917074cbe6e P 17a8ebde02364146967673d58b1560f4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:16:09.316212 22419 raft_consensus.cc:504] T dd42e2d1eaf849af8a49a917074cbe6e P 17a8ebde02364146967673d58b1560f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17a8ebde02364146967673d58b1560f4" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 32893 } } peers { permanent_uuid: "265db3c10c97462380b7418f3bb5c283" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 46489 } } peers { permanent_uuid: "74ba4103613145e5b727b8efe7fa4d85" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 35263 } }
I20260809 02:16:09.316551 22419 leader_election.cc:288] T dd42e2d1eaf849af8a49a917074cbe6e P 17a8ebde02364146967673d58b1560f4 [CANDIDATE]: Term 1 election: Requested vote from peers 265db3c10c97462380b7418f3bb5c283 (127.15.75.130:46489), 74ba4103613145e5b727b8efe7fa4d85 (127.15.75.129:35263)
I20260809 02:16:09.316905 22120 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd42e2d1eaf849af8a49a917074cbe6e" candidate_uuid: "17a8ebde02364146967673d58b1560f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "265db3c10c97462380b7418f3bb5c283"
I20260809 02:16:09.317003 21922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd42e2d1eaf849af8a49a917074cbe6e" candidate_uuid: "17a8ebde02364146967673d58b1560f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74ba4103613145e5b727b8efe7fa4d85"
I20260809 02:16:09.317021 22120 raft_consensus.cc:3037] T dd42e2d1eaf849af8a49a917074cbe6e P 265db3c10c97462380b7418f3bb5c283 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:16:09.317139 21922 raft_consensus.cc:3037] T dd42e2d1eaf849af8a49a917074cbe6e P 74ba4103613145e5b727b8efe7fa4d85 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:16:09.318070 22120 raft_consensus.cc:2454] T dd42e2d1eaf849af8a49a917074cbe6e P 265db3c10c97462380b7418f3bb5c283 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17a8ebde02364146967673d58b1560f4 in term 1.
I20260809 02:16:09.318243 21922 raft_consensus.cc:2454] T dd42e2d1eaf849af8a49a917074cbe6e P 74ba4103613145e5b727b8efe7fa4d85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17a8ebde02364146967673d58b1560f4 in term 1.
I20260809 02:16:09.318400 22226 leader_election.cc:302] T dd42e2d1eaf849af8a49a917074cbe6e P 17a8ebde02364146967673d58b1560f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17a8ebde02364146967673d58b1560f4, 265db3c10c97462380b7418f3bb5c283; no voters:
I20260809 02:16:09.318583 22419 raft_consensus.cc:2781] T dd42e2d1eaf849af8a49a917074cbe6e P 17a8ebde02364146967673d58b1560f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:16:09.318791 22419 raft_consensus.cc:686] T dd42e2d1eaf849af8a49a917074cbe6e P 17a8ebde02364146967673d58b1560f4 [term 1 LEADER]: Becoming Leader. State: Replica: 17a8ebde02364146967673d58b1560f4, State: Running, Role: LEADER
I20260809 02:16:09.319104 22419 consensus_queue.cc:237] T dd42e2d1eaf849af8a49a917074cbe6e P 17a8ebde02364146967673d58b1560f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17a8ebde02364146967673d58b1560f4" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 32893 } } peers { permanent_uuid: "265db3c10c97462380b7418f3bb5c283" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 46489 } } peers { permanent_uuid: "74ba4103613145e5b727b8efe7fa4d85" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 35263 } }
I20260809 02:16:09.322280 21698 catalog_manager.cc:5495] T dd42e2d1eaf849af8a49a917074cbe6e P 17a8ebde02364146967673d58b1560f4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 17a8ebde02364146967673d58b1560f4 (127.15.75.131). New cstate: current_term: 1 leader_uuid: "17a8ebde02364146967673d58b1560f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17a8ebde02364146967673d58b1560f4" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 32893 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "265db3c10c97462380b7418f3bb5c283" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 46489 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74ba4103613145e5b727b8efe7fa4d85" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 35263 } health_report { overall_health: UNKNOWN } } }
I20260809 02:16:09.344041 22401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:09.340392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60054 (local address 127.15.75.131:32893)
0809 02:16:09.340551 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:09.340556 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:09.340571 (+ 15us) server_negotiation.cc:407] Connection header received
0809 02:16:09.340720 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:09.340723 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:09.340777 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:09.340896 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:16:09.341166 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:09.341676 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:09.342539 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:09.342772 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:09.342888 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:09.343209 (+ 321us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:16:09.343602 (+ 393us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:16:09.343733 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:09.343819 (+ 86us) server_negotiation.cc:299] Negotiation successful
0809 02:16:09.343876 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 02:16:09.349867 22120 raft_consensus.cc:1261] T dd42e2d1eaf849af8a49a917074cbe6e P 265db3c10c97462380b7418f3bb5c283 [term 1 FOLLOWER]: Refusing update from remote peer 17a8ebde02364146967673d58b1560f4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:16:09.349867 21922 raft_consensus.cc:1261] T dd42e2d1eaf849af8a49a917074cbe6e P 74ba4103613145e5b727b8efe7fa4d85 [term 1 FOLLOWER]: Refusing update from remote peer 17a8ebde02364146967673d58b1560f4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:16:09.350550 22425 consensus_queue.cc:1035] T dd42e2d1eaf849af8a49a917074cbe6e P 17a8ebde02364146967673d58b1560f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74ba4103613145e5b727b8efe7fa4d85" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 35263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:16:09.350724 22419 consensus_queue.cc:1035] T dd42e2d1eaf849af8a49a917074cbe6e P 17a8ebde02364146967673d58b1560f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "265db3c10c97462380b7418f3bb5c283" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 46489 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:16:09.360386 22430 mvcc.cc:205] Tried to move back new op lower bound from 7316446287249891328 to 7316446287133335552. Current Snapshot: MvccSnapshot[applied={T|T < 7316446287249891328}]
I20260809 02:16:09.361281 22429 mvcc.cc:205] Tried to move back new op lower bound from 7316446287249891328 to 7316446287133335552. Current Snapshot: MvccSnapshot[applied={T|T < 7316446287249891328 or (T in {7316446287249891328})}]
I20260809 02:16:09.367213 21698 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58808:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:16:09.367370 21698 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58808:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:16:09.370009 21698 catalog_manager.cc:5782] T 00000000000000000000000000000000 P eeeadf730e854490a92ad5687b52172c: Sending DeleteTablet for 3 replicas of tablet dd42e2d1eaf849af8a49a917074cbe6e
I20260809 02:16:09.370834 22292 tablet_service.cc:1514] Processing DeleteTablet for tablet dd42e2d1eaf849af8a49a917074cbe6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:16:09 UTC) from {username='kudu', principal='kudu/127.15.75.190@KRBTEST.COM'} at 127.0.0.1:60038
I20260809 02:16:09.370874 22093 tablet_service.cc:1514] Processing DeleteTablet for tablet dd42e2d1eaf849af8a49a917074cbe6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:16:09 UTC) from {username='kudu', principal='kudu/127.15.75.190@KRBTEST.COM'} at 127.0.0.1:46850
I20260809 02:16:09.371371 22442 tablet_replica.cc:330] T dd42e2d1eaf849af8a49a917074cbe6e P 17a8ebde02364146967673d58b1560f4: stopping tablet replica
I20260809 02:16:09.371734 22442 raft_consensus.cc:2229] T dd42e2d1eaf849af8a49a917074cbe6e P 17a8ebde02364146967673d58b1560f4 [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:16:09.371665 21891 tablet_service.cc:1514] Processing DeleteTablet for tablet dd42e2d1eaf849af8a49a917074cbe6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:16:09 UTC) from {username='kudu', principal='kudu/127.15.75.190@KRBTEST.COM'} at 127.0.0.1:40496
I20260809 02:16:09.372200 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 21805
I20260809 02:16:09.372241 22442 raft_consensus.cc:2258] T dd42e2d1eaf849af8a49a917074cbe6e P 17a8ebde02364146967673d58b1560f4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:16:09.372368 22444 tablet_replica.cc:330] T dd42e2d1eaf849af8a49a917074cbe6e P 74ba4103613145e5b727b8efe7fa4d85: stopping tablet replica
I20260809 02:16:09.372963 22443 tablet_replica.cc:330] T dd42e2d1eaf849af8a49a917074cbe6e P 265db3c10c97462380b7418f3bb5c283: stopping tablet replica
I20260809 02:16:09.373301 22443 raft_consensus.cc:2229] T dd42e2d1eaf849af8a49a917074cbe6e P 265db3c10c97462380b7418f3bb5c283 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:16:09.373674 22443 raft_consensus.cc:2258] T dd42e2d1eaf849af8a49a917074cbe6e P 265db3c10c97462380b7418f3bb5c283 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:16:09.375239 22442 ts_tablet_manager.cc:1905] T dd42e2d1eaf849af8a49a917074cbe6e P 17a8ebde02364146967673d58b1560f4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:16:09.376190 22443 ts_tablet_manager.cc:1905] T dd42e2d1eaf849af8a49a917074cbe6e P 265db3c10c97462380b7418f3bb5c283: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:16:09.378147 22442 ts_tablet_manager.cc:1918] T dd42e2d1eaf849af8a49a917074cbe6e P 17a8ebde02364146967673d58b1560f4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:16:09.378255 22442 log.cc:1196] T dd42e2d1eaf849af8a49a917074cbe6e P 17a8ebde02364146967673d58b1560f4: Deleting WAL directory at /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/minicluster-data/ts-2/wal/wals/dd42e2d1eaf849af8a49a917074cbe6e
I20260809 02:16:09.378618 22442 ts_tablet_manager.cc:1939] T dd42e2d1eaf849af8a49a917074cbe6e P 17a8ebde02364146967673d58b1560f4: Deleting consensus metadata
I20260809 02:16:09.379320 22443 ts_tablet_manager.cc:1918] T dd42e2d1eaf849af8a49a917074cbe6e P 265db3c10c97462380b7418f3bb5c283: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:16:09.379421 22443 log.cc:1196] T dd42e2d1eaf849af8a49a917074cbe6e P 265db3c10c97462380b7418f3bb5c283: Deleting WAL directory at /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241741788589-15662-0/minicluster-data/ts-1/wal/wals/dd42e2d1eaf849af8a49a917074cbe6e
I20260809 02:16:09.379731 22443 ts_tablet_manager.cc:1939] T dd42e2d1eaf849af8a49a917074cbe6e P 265db3c10c97462380b7418f3bb5c283: Deleting consensus metadata
I20260809 02:16:09.379920 21680 catalog_manager.cc:4841] TS 17a8ebde02364146967673d58b1560f4 (127.15.75.131:32893): tablet dd42e2d1eaf849af8a49a917074cbe6e (table test-table [id=4921fd1dc6be4a69894a9a7e238b8f8f]) successfully deleted
I20260809 02:16:09.380865 21684 catalog_manager.cc:4841] TS 265db3c10c97462380b7418f3bb5c283 (127.15.75.130:46489): tablet dd42e2d1eaf849af8a49a917074cbe6e (table test-table [id=4921fd1dc6be4a69894a9a7e238b8f8f]) successfully deleted
W20260809 02:16:09.385098 21682 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.75.129:35263 (error 108)
I20260809 02:16:09.385412 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 22004
I20260809 02:16:09.389961 22402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:09.389616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.129:35263 (local address 127.0.0.1:40504)
0809 02:16:09.389794 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:09.389873 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.129:35263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
W20260809 02:16:09.390118 21682 catalog_manager.cc:4584] TS 74ba4103613145e5b727b8efe7fa4d85 (127.15.75.129:35263): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet dd42e2d1eaf849af8a49a917074cbe6e: Network error: Client connection negotiation failed: client connection to 127.15.75.129:35263: connect: Connection refused (error 111)
I20260809 02:16:09.395335 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 22194
I20260809 02:16:09.409952 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 21656
2026-08-09T02:16:09Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2738 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-09T02:16:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:16:09Z Disabled control of system clock
I20260809 02:16:09.445426 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241741788589-15662-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241741788589-15662-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241741788589-15662-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:37999
--webserver_interface=127.15.75.190
--webserver_port=0
--builtin_ntp_servers=127.15.75.148:36889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.75.190:37999
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 02:16:09.557324 22455 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:16:09.557562 22455 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:16:09.557597 22455 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:16:09.561444 22455 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:16:09.561491 22455 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:16:09.561506 22455 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:16:09.561522 22455 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:16:09.561537 22455 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:16:09.566118 22455 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:36889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241741788589-15662-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241741788589-15662-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.75.190:37999
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:37999
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.75.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22455
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241741788589-15662-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:16:09.567046 22455 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:16:09.568292 22455 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:16:09.576736 22463 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:09.576756 22462 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:09.576745 22469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:09.576838 22455 server_base.cc:1034] running on GCE node
I20260809 02:16:09.577653 22455 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:16:09.578704 22455 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:16:09.579890 22455 hybrid_clock.cc:648] HybridClock initialized: now 1786241769579862 us; error 41 us; skew 500 ppm
I20260809 02:16:09.581753 22455 webserver.cc:460] Webserver started at http://127.15.75.190:34801/ using document root <none> and password file <none>
I20260809 02:16:09.582335 22455 fs_manager.cc:359] Metadata directory not provided
I20260809 02:16:09.582427 22455 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:16:09.582646 22455 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:16:09.584619 22455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241741788589-15662-0/minicluster-data/master-0/data/instance:
uuid: "6d7507476b68447fa86a7f49cf81a6c2"
format_stamp: "Formatted at 2026-08-09 02:16:09 on dist-test-slave-s11t"
server_key: "b11bc72eabda3d1747cd3dcede27d978"
server_key_iv: "f3912fb6cd5d06fd2a2425ae15718a32"
server_key_version: "encryptionkey@0"
I20260809 02:16:09.585235 22455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241741788589-15662-0/minicluster-data/master-0/wal/instance:
uuid: "6d7507476b68447fa86a7f49cf81a6c2"
format_stamp: "Formatted at 2026-08-09 02:16:09 on dist-test-slave-s11t"
server_key: "b11bc72eabda3d1747cd3dcede27d978"
server_key_iv: "f3912fb6cd5d06fd2a2425ae15718a32"
server_key_version: "encryptionkey@0"
I20260809 02:16:09.588665 22455 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 02:16:09.590988 22482 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:09.592146 22455 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 02:16:09.592279 22455 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241741788589-15662-0/minicluster-data/master-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241741788589-15662-0/minicluster-data/master-0/wal
uuid: "6d7507476b68447fa86a7f49cf81a6c2"
format_stamp: "Formatted at 2026-08-09 02:16:09 on dist-test-slave-s11t"
server_key: "b11bc72eabda3d1747cd3dcede27d978"
server_key_iv: "f3912fb6cd5d06fd2a2425ae15718a32"
server_key_version: "encryptionkey@0"
I20260809 02:16:09.592396 22455 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241741788589-15662-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241741788589-15662-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241741788589-15662-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:16:09.596869 22455 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:16:09.597445 22455 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:16:09.597620 22455 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:16:09.604296 22455 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.190:37999
I20260809 02:16:09.604301 22557 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.190:37999 every 8 connection(s)
I20260809 02:16:09.605211 22455 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
I20260809 02:16:09.608266 22561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:16:09.610788 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 22455
I20260809 02:16:09.610925 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241741788589-15662-0/minicluster-data/master-0/wal/instance
I20260809 02:16:09.611231 15662 external_mini_cluster.cc:1428] Setting key 9b31ed0481f0173d6de717e4f40df352
I20260809 02:16:09.614238 22561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d7507476b68447fa86a7f49cf81a6c2: Bootstrap starting.
I20260809 02:16:09.616552 22567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:09.612612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46062 (local address 127.15.75.190:37999)
0809 02:16:09.613267 (+ 655us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:09.613294 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:09.613329 (+ 35us) server_negotiation.cc:407] Connection header received
0809 02:16:09.613925 (+ 596us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:09.613940 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:09.614262 (+ 322us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:09.614589 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:09.615360 (+ 771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:09.615365 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:09.615375 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:16:09.615386 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:09.615439 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:16:09.615455 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:09.615561 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:09.615662 (+ 101us) server_negotiation.cc:299] Negotiation successful
0809 02:16:09.615904 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":159,"threads_started":1}
I20260809 02:16:09.616758 22561 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6d7507476b68447fa86a7f49cf81a6c2: Neither blocks nor log segments found. Creating new log.
I20260809 02:16:09.617479 22561 log.cc:824] T 00000000000000000000000000000000 P 6d7507476b68447fa86a7f49cf81a6c2: Log is configured to *not* fsync() on all Append() calls
I20260809 02:16:09.619181 22561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d7507476b68447fa86a7f49cf81a6c2: No bootstrap required, opened a new log
I20260809 02:16:09.622541 22561 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6d7507476b68447fa86a7f49cf81a6c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d7507476b68447fa86a7f49cf81a6c2" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 37999 } }
I20260809 02:16:09.622776 22561 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6d7507476b68447fa86a7f49cf81a6c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:16:09.622848 22561 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6d7507476b68447fa86a7f49cf81a6c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d7507476b68447fa86a7f49cf81a6c2, State: Initialized, Role: FOLLOWER
I20260809 02:16:09.623237 22561 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6d7507476b68447fa86a7f49cf81a6c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d7507476b68447fa86a7f49cf81a6c2" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 37999 } }
I20260809 02:16:09.623376 22561 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6d7507476b68447fa86a7f49cf81a6c2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:16:09.623459 22561 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6d7507476b68447fa86a7f49cf81a6c2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:16:09.623570 22561 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6d7507476b68447fa86a7f49cf81a6c2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:16:09.624364 22561 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6d7507476b68447fa86a7f49cf81a6c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d7507476b68447fa86a7f49cf81a6c2" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 37999 } }
I20260809 02:16:09.624723 22561 leader_election.cc:302] T 00000000000000000000000000000000 P 6d7507476b68447fa86a7f49cf81a6c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6d7507476b68447fa86a7f49cf81a6c2; no voters:
I20260809 02:16:09.625037 22561 leader_election.cc:288] T 00000000000000000000000000000000 P 6d7507476b68447fa86a7f49cf81a6c2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:16:09.625133 22572 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6d7507476b68447fa86a7f49cf81a6c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:16:09.625362 22572 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6d7507476b68447fa86a7f49cf81a6c2 [term 1 LEADER]: Becoming Leader. State: Replica: 6d7507476b68447fa86a7f49cf81a6c2, State: Running, Role: LEADER
I20260809 02:16:09.625667 22572 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6d7507476b68447fa86a7f49cf81a6c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d7507476b68447fa86a7f49cf81a6c2" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 37999 } }
I20260809 02:16:09.626189 22561 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6d7507476b68447fa86a7f49cf81a6c2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:16:09.627573 22573 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d7507476b68447fa86a7f49cf81a6c2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6d7507476b68447fa86a7f49cf81a6c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d7507476b68447fa86a7f49cf81a6c2" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 37999 } } }
I20260809 02:16:09.627553 22574 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d7507476b68447fa86a7f49cf81a6c2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6d7507476b68447fa86a7f49cf81a6c2. Latest consensus state: current_term: 1 leader_uuid: "6d7507476b68447fa86a7f49cf81a6c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d7507476b68447fa86a7f49cf81a6c2" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 37999 } } }
I20260809 02:16:09.627699 22573 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d7507476b68447fa86a7f49cf81a6c2 [sys.catalog]: This master's current role is: LEADER
I20260809 02:16:09.627771 22574 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d7507476b68447fa86a7f49cf81a6c2 [sys.catalog]: This master's current role is: LEADER
I20260809 02:16:09.628180 22584 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:16:09.631121 22584 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:16:09.636056 22584 catalog_manager.cc:1347] Generated new cluster ID: af6a5f4afe724c1d9de42492859b4d60
I20260809 02:16:09.636116 22584 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:16:09.663157 22584 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:16:09.663499 22584 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:16:09.670308 22584 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6d7507476b68447fa86a7f49cf81a6c2: Generated new TSK 0
I20260809 02:16:09.670957 22584 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:16:09.679133 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241741788589-15662-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241741788589-15662-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241741788589-15662-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.129:0
--local_ip_for_outbound_sockets=127.15.75.129
--webserver_interface=127.15.75.129
--webserver_port=0
--tserver_master_addrs=127.15.75.190:37999
--builtin_ntp_servers=127.15.75.148:36889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 02:16:09.797279 22600 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:16:09.797523 22600 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:16:09.797560 22600 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:16:09.801465 22600 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:16:09.801512 22600 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:16:09.801564 22600 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.129
I20260809 02:16:09.806097 22600 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:36889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241741788589-15662-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241741788589-15662-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.75.129
--webserver_port=0
--tserver_master_addrs=127.15.75.190:37999
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22600
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.129
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241741788589-15662-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:16:09.807058 22600 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:16:09.808419 22600 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:16:09.818233 22610 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:09.818229 22608 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:09.818238 22607 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:09.818419 22600 server_base.cc:1034] running on GCE node
I20260809 02:16:09.818986 22600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:16:09.819607 22600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:16:09.820760 22600 hybrid_clock.cc:648] HybridClock initialized: now 1786241769820743 us; error 26 us; skew 500 ppm
I20260809 02:16:09.822613 22600 webserver.cc:460] Webserver started at http://127.15.75.129:39351/ using document root <none> and password file <none>
I20260809 02:16:09.823204 22600 fs_manager.cc:359] Metadata directory not provided
I20260809 02:16:09.823295 22600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:16:09.823506 22600 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:16:09.825598 22600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241741788589-15662-0/minicluster-data/ts-0/data/instance:
uuid: "8c50207057384e11affa724d35ca8768"
format_stamp: "Formatted at 2026-08-09 02:16:09 on dist-test-slave-s11t"
server_key: "1eb25e2fe0d9d978c465c51898fece8b"
server_key_iv: "71001cee679f09c654065c8f4ee73c5c"
server_key_version: "encryptionkey@0"
I20260809 02:16:09.826239 22600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241741788589-15662-0/minicluster-data/ts-0/wal/instance:
uuid: "8c50207057384e11affa724d35ca8768"
format_stamp: "Formatted at 2026-08-09 02:16:09 on dist-test-slave-s11t"
server_key: "1eb25e2fe0d9d978c465c51898fece8b"
server_key_iv: "71001cee679f09c654065c8f4ee73c5c"
server_key_version: "encryptionkey@0"
I20260809 02:16:09.830159 22600 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260809 02:16:09.832640 22619 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:09.833901 22600 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 02:16:09.834033 22600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241741788589-15662-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241741788589-15662-0/minicluster-data/ts-0/wal
uuid: "8c50207057384e11affa724d35ca8768"
format_stamp: "Formatted at 2026-08-09 02:16:09 on dist-test-slave-s11t"
server_key: "1eb25e2fe0d9d978c465c51898fece8b"
server_key_iv: "71001cee679f09c654065c8f4ee73c5c"
server_key_version: "encryptionkey@0"
I20260809 02:16:09.834139 22600 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241741788589-15662-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241741788589-15662-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241741788589-15662-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:16:09.838991 22600 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:16:09.839581 22600 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:16:09.839777 22600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:16:09.840299 22600 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:16:09.841246 22600 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:16:09.841313 22600 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:09.841382 22600 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:16:09.841427 22600 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:09.850455 22600 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.129:44151
I20260809 02:16:09.850481 22792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.129:44151 every 8 connection(s)
I20260809 02:16:09.851307 22600 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
I20260809 02:16:09.856995 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 22600
I20260809 02:16:09.857137 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241741788589-15662-0/minicluster-data/ts-0/wal/instance
I20260809 02:16:09.857431 15662 external_mini_cluster.cc:1428] Setting key 34987405caf3f352ee4fef32b2d4e4a1
I20260809 02:16:09.857424 22567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:09.853101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.129:43695 (local address 127.15.75.190:37999)
0809 02:16:09.853230 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:09.853233 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:09.854228 (+ 995us) server_negotiation.cc:407] Connection header received
0809 02:16:09.855649 (+ 1421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:09.855653 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:09.855699 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:09.855806 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:09.856950 (+ 1144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:09.856954 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:09.856957 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:16:09.856969 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:09.856991 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:16:09.857000 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:09.857100 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:09.857245 (+ 145us) server_negotiation.cc:299] Negotiation successful
0809 02:16:09.857289 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 02:16:09.858325 22797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:09.853429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:37999 (local address 127.15.75.129:43695)
0809 02:16:09.854068 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:09.854107 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:09.855433 (+ 1326us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:09.855974 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:09.855980 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:09.856064 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:09.856640 (+ 576us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:16:09.856641 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:09.856727 (+ 86us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:16:09.856809 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:16:09.857084 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:16:09.857093 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:16:09.857099 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 02:16:09.857283 (+ 184us) client_negotiation.cc:240] Negotiation successful
0809 02:16:09.857508 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":110,"threads_started":1}
I20260809 02:16:09.859421 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241741788589-15662-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241741788589-15662-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241741788589-15662-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.130:0
--local_ip_for_outbound_sockets=127.15.75.130
--webserver_interface=127.15.75.130
--webserver_port=0
--tserver_master_addrs=127.15.75.190:37999
--builtin_ntp_servers=127.15.75.148:36889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260809 02:16:09.859503 22794 heartbeater.cc:347] Connected to a master server at 127.15.75.190:37999
I20260809 02:16:09.859690 22794 heartbeater.cc:464] Registering TS with master...
I20260809 02:16:09.860073 22794 heartbeater.cc:511] Master 127.15.75.190:37999 requested a full tablet report, sending...
I20260809 02:16:09.861608 22501 ts_manager.cc:194] Registered new tserver with Master: 8c50207057384e11affa724d35ca8768 (127.15.75.129:44151)
W20260809 02:16:09.971133 22798 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:16:09.971374 22798 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:16:09.971411 22798 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:16:09.975425 22798 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:16:09.975507 22798 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:16:09.975605 22798 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.130
I20260809 02:16:09.980158 22798 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:36889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241741788589-15662-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241741788589-15662-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.75.130
--webserver_port=0
--tserver_master_addrs=127.15.75.190:37999
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22798
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.130
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241741788589-15662-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:16:09.981176 22798 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:16:09.982479 22798 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:16:09.992040 22812 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:09.992043 22805 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:09.992070 22809 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:09.992431 22798 server_base.cc:1034] running on GCE node
I20260809 02:16:09.992878 22798 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:16:09.993458 22798 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:16:09.994614 22798 hybrid_clock.cc:648] HybridClock initialized: now 1786241769994583 us; error 42 us; skew 500 ppm
I20260809 02:16:09.996424 22798 webserver.cc:460] Webserver started at http://127.15.75.130:45253/ using document root <none> and password file <none>
I20260809 02:16:09.997049 22798 fs_manager.cc:359] Metadata directory not provided
I20260809 02:16:09.997112 22798 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:16:09.997342 22798 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:16:09.999297 22798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241741788589-15662-0/minicluster-data/ts-1/data/instance:
uuid: "6f4a6204162d44b09eeac13b31e546e6"
format_stamp: "Formatted at 2026-08-09 02:16:09 on dist-test-slave-s11t"
server_key: "f1ea327fb55486776e89f4811c7a0856"
server_key_iv: "2affd577609ac4ff6954336982cc681a"
server_key_version: "encryptionkey@0"
I20260809 02:16:09.999833 22798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241741788589-15662-0/minicluster-data/ts-1/wal/instance:
uuid: "6f4a6204162d44b09eeac13b31e546e6"
format_stamp: "Formatted at 2026-08-09 02:16:09 on dist-test-slave-s11t"
server_key: "f1ea327fb55486776e89f4811c7a0856"
server_key_iv: "2affd577609ac4ff6954336982cc681a"
server_key_version: "encryptionkey@0"
I20260809 02:16:10.003500 22798 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260809 02:16:10.005990 22822 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:10.007186 22798 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 02:16:10.007326 22798 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241741788589-15662-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241741788589-15662-0/minicluster-data/ts-1/wal
uuid: "6f4a6204162d44b09eeac13b31e546e6"
format_stamp: "Formatted at 2026-08-09 02:16:09 on dist-test-slave-s11t"
server_key: "f1ea327fb55486776e89f4811c7a0856"
server_key_iv: "2affd577609ac4ff6954336982cc681a"
server_key_version: "encryptionkey@0"
I20260809 02:16:10.007434 22798 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241741788589-15662-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241741788589-15662-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241741788589-15662-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:16:10.012009 22798 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:16:10.012581 22798 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:16:10.012787 22798 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:16:10.013319 22798 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:16:10.014184 22798 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:16:10.014262 22798 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:10.014328 22798 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:16:10.014374 22798 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:10.023218 22798 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.130:41709
I20260809 02:16:10.023236 23004 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.130:41709 every 8 connection(s)
I20260809 02:16:10.024012 22798 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
I20260809 02:16:10.026393 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 22798
I20260809 02:16:10.026523 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241741788589-15662-0/minicluster-data/ts-1/wal/instance
I20260809 02:16:10.026898 15662 external_mini_cluster.cc:1428] Setting key dbc018559f7eac5d44a3deab3650227c
I20260809 02:16:10.028954 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241741788589-15662-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241741788589-15662-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241741788589-15662-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.131:0
--local_ip_for_outbound_sockets=127.15.75.131
--webserver_interface=127.15.75.131
--webserver_port=0
--tserver_master_addrs=127.15.75.190:37999
--builtin_ntp_servers=127.15.75.148:36889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260809 02:16:10.030309 22567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:10.025732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.130:33331 (local address 127.15.75.190:37999)
0809 02:16:10.025863 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:10.025867 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:10.026971 (+ 1104us) server_negotiation.cc:407] Connection header received
0809 02:16:10.028431 (+ 1460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:10.028435 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:10.028475 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:10.028564 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:10.029808 (+ 1244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:10.029814 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:10.029820 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:16:10.029838 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:10.029865 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:16:10.029878 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:10.029964 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:10.030071 (+ 107us) server_negotiation.cc:299] Negotiation successful
0809 02:16:10.030117 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 02:16:10.031194 23009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:10.026072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:37999 (local address 127.15.75.130:33331)
0809 02:16:10.026824 (+ 752us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:10.026873 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:10.028246 (+ 1373us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:10.028788 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:10.028794 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:10.028886 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:10.029504 (+ 618us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:16:10.029507 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:10.029561 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:16:10.029627 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:16:10.029966 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:16:10.029973 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:16:10.029977 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:16:10.030100 (+ 123us) client_negotiation.cc:240] Negotiation successful
0809 02:16:10.030275 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":185,"threads_started":1}
I20260809 02:16:10.032262 23007 heartbeater.cc:347] Connected to a master server at 127.15.75.190:37999
I20260809 02:16:10.032486 23007 heartbeater.cc:464] Registering TS with master...
I20260809 02:16:10.032903 23007 heartbeater.cc:511] Master 127.15.75.190:37999 requested a full tablet report, sending...
I20260809 02:16:10.033914 22501 ts_manager.cc:194] Registered new tserver with Master: 6f4a6204162d44b09eeac13b31e546e6 (127.15.75.130:41709)
W20260809 02:16:10.145841 23010 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:16:10.146077 23010 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:16:10.146116 23010 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:16:10.150199 23010 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:16:10.150244 23010 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:16:10.150298 23010 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.131
I20260809 02:16:10.154899 23010 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:36889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241741788589-15662-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241741788589-15662-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.75.131
--webserver_port=0
--tserver_master_addrs=127.15.75.190:37999
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23010
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.131
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241741788589-15662-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:16:10.155809 23010 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:16:10.157197 23010 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:16:10.166591 23023 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:10.166638 23019 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:10.166802 23010 server_base.cc:1034] running on GCE node
W20260809 02:16:10.166607 23021 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:10.167335 23010 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:16:10.167958 23010 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:16:10.169142 23010 hybrid_clock.cc:648] HybridClock initialized: now 1786241770169127 us; error 25 us; skew 500 ppm
I20260809 02:16:10.173889 23010 webserver.cc:460] Webserver started at http://127.15.75.131:39763/ using document root <none> and password file <none>
I20260809 02:16:10.174427 23010 fs_manager.cc:359] Metadata directory not provided
I20260809 02:16:10.174492 23010 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:16:10.174660 23010 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:16:10.176654 23010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241741788589-15662-0/minicluster-data/ts-2/data/instance:
uuid: "c6d25d1fc6294278a9c7ebdcda916e91"
format_stamp: "Formatted at 2026-08-09 02:16:10 on dist-test-slave-s11t"
server_key: "be2e35b80fbb0bf46f7b06239fbe9bdf"
server_key_iv: "1f0df8f88663eca7ef711f6f9e9bf8eb"
server_key_version: "encryptionkey@0"
I20260809 02:16:10.177276 23010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241741788589-15662-0/minicluster-data/ts-2/wal/instance:
uuid: "c6d25d1fc6294278a9c7ebdcda916e91"
format_stamp: "Formatted at 2026-08-09 02:16:10 on dist-test-slave-s11t"
server_key: "be2e35b80fbb0bf46f7b06239fbe9bdf"
server_key_iv: "1f0df8f88663eca7ef711f6f9e9bf8eb"
server_key_version: "encryptionkey@0"
I20260809 02:16:10.180776 23010 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.002s
I20260809 02:16:10.183197 23031 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:10.184463 23010 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 02:16:10.184604 23010 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241741788589-15662-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241741788589-15662-0/minicluster-data/ts-2/wal
uuid: "c6d25d1fc6294278a9c7ebdcda916e91"
format_stamp: "Formatted at 2026-08-09 02:16:10 on dist-test-slave-s11t"
server_key: "be2e35b80fbb0bf46f7b06239fbe9bdf"
server_key_iv: "1f0df8f88663eca7ef711f6f9e9bf8eb"
server_key_version: "encryptionkey@0"
I20260809 02:16:10.184751 23010 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241741788589-15662-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241741788589-15662-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241741788589-15662-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:16:10.189208 23010 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:16:10.189793 23010 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:16:10.189968 23010 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:16:10.190500 23010 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:16:10.191380 23010 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:16:10.191447 23010 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:10.191510 23010 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:16:10.191557 23010 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:10.200891 23010 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.131:39833
I20260809 02:16:10.200922 23199 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.131:39833 every 8 connection(s)
I20260809 02:16:10.201735 23010 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
I20260809 02:16:10.207692 22567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:10.203489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.131:59593 (local address 127.15.75.190:37999)
0809 02:16:10.203638 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:10.203643 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:10.204812 (+ 1169us) server_negotiation.cc:407] Connection header received
0809 02:16:10.206110 (+ 1298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:10.206113 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:10.206152 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:10.206227 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:10.207251 (+ 1024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:10.207254 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:10.207256 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:16:10.207267 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:10.207287 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:16:10.207296 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:10.207375 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:10.207547 (+ 172us) server_negotiation.cc:299] Negotiation successful
0809 02:16:10.207586 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 02:16:10.207881 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 23010
I20260809 02:16:10.208009 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241741788589-15662-0/minicluster-data/ts-2/wal/instance
I20260809 02:16:10.208335 15662 external_mini_cluster.cc:1428] Setting key 94041f92259121de45512c09b594b1f5
I20260809 02:16:10.208868 23203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:10.203923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:37999 (local address 127.15.75.131:59593)
0809 02:16:10.204605 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:10.204646 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:10.205921 (+ 1275us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:10.206391 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:10.206398 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:10.206483 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:10.206999 (+ 516us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:16:10.207001 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:10.207041 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:16:10.207109 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:16:10.207404 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:16:10.207413 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:16:10.207418 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 02:16:10.207599 (+ 181us) client_negotiation.cc:240] Negotiation successful
0809 02:16:10.207844 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":109,"threads_started":1}
I20260809 02:16:10.210664 23200 heartbeater.cc:347] Connected to a master server at 127.15.75.190:37999
I20260809 02:16:10.211081 23200 heartbeater.cc:464] Registering TS with master...
I20260809 02:16:10.211573 23200 heartbeater.cc:511] Master 127.15.75.190:37999 requested a full tablet report, sending...
I20260809 02:16:10.212812 22501 ts_manager.cc:194] Registered new tserver with Master: c6d25d1fc6294278a9c7ebdcda916e91 (127.15.75.131:39833)
I20260809 02:16:10.223634 15662 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 02:16:10.227715 22567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:10.225803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46064 (local address 127.15.75.190:37999)
0809 02:16:10.225969 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:10.225975 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:10.226157 (+ 182us) server_negotiation.cc:407] Connection header received
0809 02:16:10.226315 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:10.226318 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:10.226367 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:10.226451 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:10.227216 (+ 765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:10.227220 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:10.227222 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:16:10.227234 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:10.227256 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:16:10.227270 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:10.227392 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:10.227538 (+ 146us) server_negotiation.cc:299] Negotiation successful
0809 02:16:10.227582 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 02:16:10.232223 22501 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46064:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:16:10.236152 22501 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:16:10.255959 23223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:10.252914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.131:39833 (local address 127.0.0.1:40720)
0809 02:16:10.253577 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:10.253591 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:10.253666 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:10.254120 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:10.254123 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:10.254137 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:10.255279 (+ 1142us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:16:10.255281 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:10.255291 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:16:10.255320 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:16:10.255610 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:16:10.255620 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:16:10.255626 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 02:16:10.255742 (+ 116us) client_negotiation.cc:240] Negotiation successful
0809 02:16:10.255816 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":102,"threads_started":1}
I20260809 02:16:10.256081 23221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:10.251901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43772 (local address 127.15.75.129:44151)
0809 02:16:10.252213 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:10.252228 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:10.252257 (+ 29us) server_negotiation.cc:407] Connection header received
0809 02:16:10.252366 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:10.252373 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:10.252562 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:10.252673 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:10.255204 (+ 2531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:10.255215 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:10.255230 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:16:10.255247 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:10.255301 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:16:10.255355 (+ 54us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:10.255472 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:10.255830 (+ 358us) server_negotiation.cc:299] Negotiation successful
0809 02:16:10.255909 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":117,"threads_started":1}
I20260809 02:16:10.256494 23219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:10.251730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.129:44151 (local address 127.0.0.1:43772)
0809 02:16:10.252096 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:10.252142 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:10.252238 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:10.252739 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:10.252745 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:10.252824 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:10.254974 (+ 2150us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:16:10.254976 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:10.254998 (+ 22us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:16:10.255075 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:16:10.255674 (+ 599us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:16:10.255678 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:16:10.255683 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 02:16:10.256295 (+ 612us) client_negotiation.cc:240] Negotiation successful
0809 02:16:10.256334 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":96,"threads_started":1}
I20260809 02:16:10.256577 23222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:10.253205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.130:41709 (local address 127.0.0.1:34398)
0809 02:16:10.253845 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:10.253855 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:10.253945 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:10.254348 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:10.254351 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:10.254364 (+ 13us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:10.255565 (+ 1201us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:16:10.255568 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:10.255582 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:16:10.255617 (+ 35us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:16:10.256070 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:16:10.256073 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:16:10.256076 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 02:16:10.256143 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 02:16:10.256177 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":108,"threads_started":1}
I20260809 02:16:10.256700 23226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:10.253348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40720 (local address 127.15.75.131:39833)
0809 02:16:10.253717 (+ 369us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:10.253729 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:10.253753 (+ 24us) server_negotiation.cc:407] Connection header received
0809 02:16:10.253808 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:10.253813 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:10.253980 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:10.254125 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:10.255435 (+ 1310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:10.255441 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:10.255452 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:16:10.255464 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:10.255510 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:16:10.255529 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:10.255636 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:10.256316 (+ 680us) server_negotiation.cc:299] Negotiation successful
0809 02:16:10.256387 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":186,"threads_started":1}
I20260809 02:16:10.259435 22699 tablet_service.cc:1467] Processing CreateTablet for tablet b3f521c1d7344eb3a6bbf3f8e8ea485a (DEFAULT_TABLE table=test-table [id=97441ab009004a67806832e400ce42ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:16:10.259864 23105 tablet_service.cc:1467] Processing CreateTablet for tablet b3f521c1d7344eb3a6bbf3f8e8ea485a (DEFAULT_TABLE table=test-table [id=97441ab009004a67806832e400ce42ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:16:10.260557 22699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3f521c1d7344eb3a6bbf3f8e8ea485a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:16:10.260902 23227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:10.253410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34398 (local address 127.15.75.130:41709)
0809 02:16:10.253858 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:10.253868 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:10.253933 (+ 65us) server_negotiation.cc:407] Connection header received
0809 02:16:10.254056 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:10.254064 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:10.254217 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:10.254344 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:10.255775 (+ 1431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:10.255784 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:10.255800 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:16:10.255814 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:10.255877 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:16:10.255913 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:10.256023 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:10.256761 (+ 738us) server_negotiation.cc:299] Negotiation successful
0809 02:16:10.256846 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":138,"threads_started":1}
I20260809 02:16:10.260998 23105 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3f521c1d7344eb3a6bbf3f8e8ea485a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:16:10.264055 22909 tablet_service.cc:1467] Processing CreateTablet for tablet b3f521c1d7344eb3a6bbf3f8e8ea485a (DEFAULT_TABLE table=test-table [id=97441ab009004a67806832e400ce42ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:16:10.265220 22909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3f521c1d7344eb3a6bbf3f8e8ea485a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:16:10.268419 23228 tablet_bootstrap.cc:492] T b3f521c1d7344eb3a6bbf3f8e8ea485a P 8c50207057384e11affa724d35ca8768: Bootstrap starting.
I20260809 02:16:10.270852 23228 tablet_bootstrap.cc:654] T b3f521c1d7344eb3a6bbf3f8e8ea485a P 8c50207057384e11affa724d35ca8768: Neither blocks nor log segments found. Creating new log.
I20260809 02:16:10.271626 23229 tablet_bootstrap.cc:492] T b3f521c1d7344eb3a6bbf3f8e8ea485a P c6d25d1fc6294278a9c7ebdcda916e91: Bootstrap starting.
I20260809 02:16:10.271778 23228 log.cc:824] T b3f521c1d7344eb3a6bbf3f8e8ea485a P 8c50207057384e11affa724d35ca8768: Log is configured to *not* fsync() on all Append() calls
I20260809 02:16:10.272969 23232 tablet_bootstrap.cc:492] T b3f521c1d7344eb3a6bbf3f8e8ea485a P 6f4a6204162d44b09eeac13b31e546e6: Bootstrap starting.
I20260809 02:16:10.274091 23229 tablet_bootstrap.cc:654] T b3f521c1d7344eb3a6bbf3f8e8ea485a P c6d25d1fc6294278a9c7ebdcda916e91: Neither blocks nor log segments found. Creating new log.
I20260809 02:16:10.274718 23228 tablet_bootstrap.cc:492] T b3f521c1d7344eb3a6bbf3f8e8ea485a P 8c50207057384e11affa724d35ca8768: No bootstrap required, opened a new log
I20260809 02:16:10.274812 23232 tablet_bootstrap.cc:654] T b3f521c1d7344eb3a6bbf3f8e8ea485a P 6f4a6204162d44b09eeac13b31e546e6: Neither blocks nor log segments found. Creating new log.
I20260809 02:16:10.275031 23228 ts_tablet_manager.cc:1397] T b3f521c1d7344eb3a6bbf3f8e8ea485a P 8c50207057384e11affa724d35ca8768: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260809 02:16:10.275025 23229 log.cc:824] T b3f521c1d7344eb3a6bbf3f8e8ea485a P c6d25d1fc6294278a9c7ebdcda916e91: Log is configured to *not* fsync() on all Append() calls
I20260809 02:16:10.275645 23232 log.cc:824] T b3f521c1d7344eb3a6bbf3f8e8ea485a P 6f4a6204162d44b09eeac13b31e546e6: Log is configured to *not* fsync() on all Append() calls
I20260809 02:16:10.277336 23229 tablet_bootstrap.cc:492] T b3f521c1d7344eb3a6bbf3f8e8ea485a P c6d25d1fc6294278a9c7ebdcda916e91: No bootstrap required, opened a new log
I20260809 02:16:10.277539 23229 ts_tablet_manager.cc:1397] T b3f521c1d7344eb3a6bbf3f8e8ea485a P c6d25d1fc6294278a9c7ebdcda916e91: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260809 02:16:10.278218 23232 tablet_bootstrap.cc:492] T b3f521c1d7344eb3a6bbf3f8e8ea485a P 6f4a6204162d44b09eeac13b31e546e6: No bootstrap required, opened a new log
I20260809 02:16:10.278415 23232 ts_tablet_manager.cc:1397] T b3f521c1d7344eb3a6bbf3f8e8ea485a P 6f4a6204162d44b09eeac13b31e546e6: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260809 02:16:10.279125 23228 raft_consensus.cc:348] T b3f521c1d7344eb3a6bbf3f8e8ea485a P 8c50207057384e11affa724d35ca8768 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c50207057384e11affa724d35ca8768" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 44151 } } peers { permanent_uuid: "c6d25d1fc6294278a9c7ebdcda916e91" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 39833 } } peers { permanent_uuid: "6f4a6204162d44b09eeac13b31e546e6" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 41709 } }
I20260809 02:16:10.279459 23228 raft_consensus.cc:374] T b3f521c1d7344eb3a6bbf3f8e8ea485a P 8c50207057384e11affa724d35ca8768 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:16:10.279516 23228 raft_consensus.cc:729] T b3f521c1d7344eb3a6bbf3f8e8ea485a P 8c50207057384e11affa724d35ca8768 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c50207057384e11affa724d35ca8768, State: Initialized, Role: FOLLOWER
I20260809 02:16:10.280009 23228 consensus_queue.cc:260] T b3f521c1d7344eb3a6bbf3f8e8ea485a P 8c50207057384e11affa724d35ca8768 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c50207057384e11affa724d35ca8768" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 44151 } } peers { permanent_uuid: "c6d25d1fc6294278a9c7ebdcda916e91" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 39833 } } peers { permanent_uuid: "6f4a6204162d44b09eeac13b31e546e6" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 41709 } }
I20260809 02:16:10.280956 23228 ts_tablet_manager.cc:1428] T b3f521c1d7344eb3a6bbf3f8e8ea485a P 8c50207057384e11affa724d35ca8768: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 02:16:10.281324 22794 heartbeater.cc:503] Master 127.15.75.190:37999 was elected leader, sending a full tablet report...
I20260809 02:16:10.282162 23229 raft_consensus.cc:348] T b3f521c1d7344eb3a6bbf3f8e8ea485a P c6d25d1fc6294278a9c7ebdcda916e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c50207057384e11affa724d35ca8768" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 44151 } } peers { permanent_uuid: "c6d25d1fc6294278a9c7ebdcda916e91" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 39833 } } peers { permanent_uuid: "6f4a6204162d44b09eeac13b31e546e6" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 41709 } }
I20260809 02:16:10.282431 23229 raft_consensus.cc:374] T b3f521c1d7344eb3a6bbf3f8e8ea485a P c6d25d1fc6294278a9c7ebdcda916e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:16:10.282507 23229 raft_consensus.cc:729] T b3f521c1d7344eb3a6bbf3f8e8ea485a P c6d25d1fc6294278a9c7ebdcda916e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6d25d1fc6294278a9c7ebdcda916e91, State: Initialized, Role: FOLLOWER
I20260809 02:16:10.282939 23229 consensus_queue.cc:260] T b3f521c1d7344eb3a6bbf3f8e8ea485a P c6d25d1fc6294278a9c7ebdcda916e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c50207057384e11affa724d35ca8768" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 44151 } } peers { permanent_uuid: "c6d25d1fc6294278a9c7ebdcda916e91" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 39833 } } peers { permanent_uuid: "6f4a6204162d44b09eeac13b31e546e6" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 41709 } }
I20260809 02:16:10.282954 23232 raft_consensus.cc:348] T b3f521c1d7344eb3a6bbf3f8e8ea485a P 6f4a6204162d44b09eeac13b31e546e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c50207057384e11affa724d35ca8768" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 44151 } } peers { permanent_uuid: "c6d25d1fc6294278a9c7ebdcda916e91" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 39833 } } peers { permanent_uuid: "6f4a6204162d44b09eeac13b31e546e6" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 41709 } }
I20260809 02:16:10.283193 23232 raft_consensus.cc:374] T b3f521c1d7344eb3a6bbf3f8e8ea485a P 6f4a6204162d44b09eeac13b31e546e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:16:10.283265 23232 raft_consensus.cc:729] T b3f521c1d7344eb3a6bbf3f8e8ea485a P 6f4a6204162d44b09eeac13b31e546e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f4a6204162d44b09eeac13b31e546e6, State: Initialized, Role: FOLLOWER
I20260809 02:16:10.283666 23232 consensus_queue.cc:260] T b3f521c1d7344eb3a6bbf3f8e8ea485a P 6f4a6204162d44b09eeac13b31e546e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c50207057384e11affa724d35ca8768" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 44151 } } peers { permanent_uuid: "c6d25d1fc6294278a9c7ebdcda916e91" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 39833 } } peers { permanent_uuid: "6f4a6204162d44b09eeac13b31e546e6" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 41709 } }
I20260809 02:16:10.283972 23229 ts_tablet_manager.cc:1428] T b3f521c1d7344eb3a6bbf3f8e8ea485a P c6d25d1fc6294278a9c7ebdcda916e91: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20260809 02:16:10.284319 23200 heartbeater.cc:503] Master 127.15.75.190:37999 was elected leader, sending a full tablet report...
I20260809 02:16:10.284512 23007 heartbeater.cc:503] Master 127.15.75.190:37999 was elected leader, sending a full tablet report...
I20260809 02:16:10.284618 23232 ts_tablet_manager.cc:1428] T b3f521c1d7344eb3a6bbf3f8e8ea485a P 6f4a6204162d44b09eeac13b31e546e6: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
W20260809 02:16:10.352459 22796 tablet.cc:2369] T b3f521c1d7344eb3a6bbf3f8e8ea485a P 8c50207057384e11affa724d35ca8768: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 02:16:10.452775 23201 tablet.cc:2369] T b3f521c1d7344eb3a6bbf3f8e8ea485a P c6d25d1fc6294278a9c7ebdcda916e91: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:16:10.530467 23239 raft_consensus.cc:482] T b3f521c1d7344eb3a6bbf3f8e8ea485a P 6f4a6204162d44b09eeac13b31e546e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:16:10.530732 23239 raft_consensus.cc:504] T b3f521c1d7344eb3a6bbf3f8e8ea485a P 6f4a6204162d44b09eeac13b31e546e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c50207057384e11affa724d35ca8768" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 44151 } } peers { permanent_uuid: "c6d25d1fc6294278a9c7ebdcda916e91" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 39833 } } peers { permanent_uuid: "6f4a6204162d44b09eeac13b31e546e6" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 41709 } }
W20260809 02:16:10.531070 23008 tablet.cc:2369] T b3f521c1d7344eb3a6bbf3f8e8ea485a P 6f4a6204162d44b09eeac13b31e546e6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:16:10.532001 23239 leader_election.cc:288] T b3f521c1d7344eb3a6bbf3f8e8ea485a P 6f4a6204162d44b09eeac13b31e546e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c50207057384e11affa724d35ca8768 (127.15.75.129:44151), c6d25d1fc6294278a9c7ebdcda916e91 (127.15.75.131:39833)
I20260809 02:16:10.534235 23221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:10.532403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.130:56763 (local address 127.15.75.129:44151)
0809 02:16:10.532553 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:10.532560 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:10.532791 (+ 231us) server_negotiation.cc:407] Connection header received
0809 02:16:10.532889 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:10.532892 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:10.532939 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:10.533029 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:10.533750 (+ 721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:10.533755 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:10.533762 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:16:10.533785 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:10.533811 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:16:10.533822 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:10.533905 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:10.534035 (+ 130us) server_negotiation.cc:299] Negotiation successful
0809 02:16:10.534084 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 02:16:10.534241 23240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:10.532268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.129:44151 (local address 127.15.75.130:56763)
0809 02:16:10.532641 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:10.532659 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:10.532809 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:10.533062 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:10.533069 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:10.533088 (+ 19us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:10.533583 (+ 495us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:16:10.533584 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:10.533594 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:16:10.533619 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:16:10.533917 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:16:10.533923 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:16:10.533929 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 02:16:10.534022 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 02:16:10.534068 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":76,"threads_started":1}
I20260809 02:16:10.534721 23241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:10.532268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.131:39833 (local address 127.15.75.130:45255)
0809 02:16:10.532774 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:10.532789 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:10.532894 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:10.533590 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:10.533593 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:10.533606 (+ 13us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:10.534098 (+ 492us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:16:10.534100 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:10.534111 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:16:10.534137 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:16:10.534470 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:16:10.534473 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:16:10.534476 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 02:16:10.534555 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 02:16:10.534590 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":397,"mutex_wait_us":161,"thread_start_us":100,"threads_started":1}
I20260809 02:16:10.534718 23226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:10.532498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.130:45255 (local address 127.15.75.131:39833)
0809 02:16:10.532639 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:10.532644 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:10.533119 (+ 475us) server_negotiation.cc:407] Connection header received
0809 02:16:10.533164 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:10.533167 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:10.533200 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:10.533292 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:10.534269 (+ 977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:10.534274 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:10.534278 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:16:10.534293 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:10.534317 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:16:10.534328 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:10.534426 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:10.534562 (+ 136us) server_negotiation.cc:299] Negotiation successful
0809 02:16:10.534603 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 02:16:10.534945 22723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3f521c1d7344eb3a6bbf3f8e8ea485a" candidate_uuid: "6f4a6204162d44b09eeac13b31e546e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c50207057384e11affa724d35ca8768" is_pre_election: true
I20260809 02:16:10.535267 22723 raft_consensus.cc:2454] T b3f521c1d7344eb3a6bbf3f8e8ea485a P 8c50207057384e11affa724d35ca8768 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f4a6204162d44b09eeac13b31e546e6 in term 0.
I20260809 02:16:10.535753 22829 leader_election.cc:302] T b3f521c1d7344eb3a6bbf3f8e8ea485a P 6f4a6204162d44b09eeac13b31e546e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f4a6204162d44b09eeac13b31e546e6, 8c50207057384e11affa724d35ca8768; no voters:
I20260809 02:16:10.536015 23134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3f521c1d7344eb3a6bbf3f8e8ea485a" candidate_uuid: "6f4a6204162d44b09eeac13b31e546e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6d25d1fc6294278a9c7ebdcda916e91" is_pre_election: true
I20260809 02:16:10.536054 23239 raft_consensus.cc:2781] T b3f521c1d7344eb3a6bbf3f8e8ea485a P 6f4a6204162d44b09eeac13b31e546e6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:16:10.536149 23239 raft_consensus.cc:482] T b3f521c1d7344eb3a6bbf3f8e8ea485a P 6f4a6204162d44b09eeac13b31e546e6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:16:10.536195 23239 raft_consensus.cc:3037] T b3f521c1d7344eb3a6bbf3f8e8ea485a P 6f4a6204162d44b09eeac13b31e546e6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:16:10.536341 23134 raft_consensus.cc:2454] T b3f521c1d7344eb3a6bbf3f8e8ea485a P c6d25d1fc6294278a9c7ebdcda916e91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f4a6204162d44b09eeac13b31e546e6 in term 0.
I20260809 02:16:10.537535 23239 raft_consensus.cc:504] T b3f521c1d7344eb3a6bbf3f8e8ea485a P 6f4a6204162d44b09eeac13b31e546e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c50207057384e11affa724d35ca8768" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 44151 } } peers { permanent_uuid: "c6d25d1fc6294278a9c7ebdcda916e91" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 39833 } } peers { permanent_uuid: "6f4a6204162d44b09eeac13b31e546e6" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 41709 } }
I20260809 02:16:10.537894 23239 leader_election.cc:288] T b3f521c1d7344eb3a6bbf3f8e8ea485a P 6f4a6204162d44b09eeac13b31e546e6 [CANDIDATE]: Term 1 election: Requested vote from peers 8c50207057384e11affa724d35ca8768 (127.15.75.129:44151), c6d25d1fc6294278a9c7ebdcda916e91 (127.15.75.131:39833)
I20260809 02:16:10.538223 22723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3f521c1d7344eb3a6bbf3f8e8ea485a" candidate_uuid: "6f4a6204162d44b09eeac13b31e546e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c50207057384e11affa724d35ca8768"
I20260809 02:16:10.538261 23134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3f521c1d7344eb3a6bbf3f8e8ea485a" candidate_uuid: "6f4a6204162d44b09eeac13b31e546e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6d25d1fc6294278a9c7ebdcda916e91"
I20260809 02:16:10.538329 22723 raft_consensus.cc:3037] T b3f521c1d7344eb3a6bbf3f8e8ea485a P 8c50207057384e11affa724d35ca8768 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:16:10.538415 23134 raft_consensus.cc:3037] T b3f521c1d7344eb3a6bbf3f8e8ea485a P c6d25d1fc6294278a9c7ebdcda916e91 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:16:10.539544 22723 raft_consensus.cc:2454] T b3f521c1d7344eb3a6bbf3f8e8ea485a P 8c50207057384e11affa724d35ca8768 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f4a6204162d44b09eeac13b31e546e6 in term 1.
I20260809 02:16:10.539659 23134 raft_consensus.cc:2454] T b3f521c1d7344eb3a6bbf3f8e8ea485a P c6d25d1fc6294278a9c7ebdcda916e91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f4a6204162d44b09eeac13b31e546e6 in term 1.
I20260809 02:16:10.539868 22829 leader_election.cc:302] T b3f521c1d7344eb3a6bbf3f8e8ea485a P 6f4a6204162d44b09eeac13b31e546e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f4a6204162d44b09eeac13b31e546e6, 8c50207057384e11affa724d35ca8768; no voters:
I20260809 02:16:10.540098 23239 raft_consensus.cc:2781] T b3f521c1d7344eb3a6bbf3f8e8ea485a P 6f4a6204162d44b09eeac13b31e546e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:16:10.540349 23239 raft_consensus.cc:686] T b3f521c1d7344eb3a6bbf3f8e8ea485a P 6f4a6204162d44b09eeac13b31e546e6 [term 1 LEADER]: Becoming Leader. State: Replica: 6f4a6204162d44b09eeac13b31e546e6, State: Running, Role: LEADER
I20260809 02:16:10.540717 23239 consensus_queue.cc:237] T b3f521c1d7344eb3a6bbf3f8e8ea485a P 6f4a6204162d44b09eeac13b31e546e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c50207057384e11affa724d35ca8768" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 44151 } } peers { permanent_uuid: "c6d25d1fc6294278a9c7ebdcda916e91" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 39833 } } peers { permanent_uuid: "6f4a6204162d44b09eeac13b31e546e6" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 41709 } }
I20260809 02:16:10.543869 22500 catalog_manager.cc:5495] T b3f521c1d7344eb3a6bbf3f8e8ea485a P 6f4a6204162d44b09eeac13b31e546e6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f4a6204162d44b09eeac13b31e546e6 (127.15.75.130). New cstate: current_term: 1 leader_uuid: "6f4a6204162d44b09eeac13b31e546e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c50207057384e11affa724d35ca8768" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 44151 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c6d25d1fc6294278a9c7ebdcda916e91" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 39833 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f4a6204162d44b09eeac13b31e546e6" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 41709 } health_report { overall_health: HEALTHY } } }
I20260809 02:16:10.618335 23227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:10.616766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34406 (local address 127.15.75.130:41709)
0809 02:16:10.616956 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:10.616962 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:10.617004 (+ 42us) server_negotiation.cc:407] Connection header received
0809 02:16:10.617100 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:10.617103 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:10.617178 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:10.617255 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:10.617958 (+ 703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:10.617962 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:10.617965 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:16:10.617979 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:10.618003 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:16:10.618013 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:10.618097 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:10.618155 (+ 58us) server_negotiation.cc:299] Negotiation successful
0809 02:16:10.618194 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260809 02:16:10.624536 23134 raft_consensus.cc:1261] T b3f521c1d7344eb3a6bbf3f8e8ea485a P c6d25d1fc6294278a9c7ebdcda916e91 [term 1 FOLLOWER]: Refusing update from remote peer 6f4a6204162d44b09eeac13b31e546e6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:16:10.624557 22723 raft_consensus.cc:1261] T b3f521c1d7344eb3a6bbf3f8e8ea485a P 8c50207057384e11affa724d35ca8768 [term 1 FOLLOWER]: Refusing update from remote peer 6f4a6204162d44b09eeac13b31e546e6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:16:10.625308 23242 consensus_queue.cc:1035] T b3f521c1d7344eb3a6bbf3f8e8ea485a P 6f4a6204162d44b09eeac13b31e546e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6d25d1fc6294278a9c7ebdcda916e91" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 39833 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:16:10.625486 23239 consensus_queue.cc:1035] T b3f521c1d7344eb3a6bbf3f8e8ea485a P 6f4a6204162d44b09eeac13b31e546e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c50207057384e11affa724d35ca8768" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 44151 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:16:10.633399 23246 mvcc.cc:205] Tried to move back new op lower bound from 7316446292470706176 to 7316446292137705472. Current Snapshot: MvccSnapshot[applied={T|T < 7316446292470706176}]
I20260809 02:16:10.636678 23247 mvcc.cc:205] Tried to move back new op lower bound from 7316446292470706176 to 7316446292137705472. Current Snapshot: MvccSnapshot[applied={T|T < 7316446292470706176}]
I20260809 02:16:10.642066 22499 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46064:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:16:10.642200 22499 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46064:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:16:10.644444 22499 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6d7507476b68447fa86a7f49cf81a6c2: Sending DeleteTablet for 3 replicas of tablet b3f521c1d7344eb3a6bbf3f8e8ea485a
I20260809 02:16:10.645236 22699 tablet_service.cc:1514] Processing DeleteTablet for tablet b3f521c1d7344eb3a6bbf3f8e8ea485a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:16:10 UTC) from {username='slave'} at 127.0.0.1:43772
I20260809 02:16:10.645221 23105 tablet_service.cc:1514] Processing DeleteTablet for tablet b3f521c1d7344eb3a6bbf3f8e8ea485a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:16:10 UTC) from {username='slave'} at 127.0.0.1:40720
I20260809 02:16:10.645686 23258 tablet_replica.cc:330] T b3f521c1d7344eb3a6bbf3f8e8ea485a P 8c50207057384e11affa724d35ca8768: stopping tablet replica
I20260809 02:16:10.645697 22909 tablet_service.cc:1514] Processing DeleteTablet for tablet b3f521c1d7344eb3a6bbf3f8e8ea485a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:16:10 UTC) from {username='slave'} at 127.0.0.1:34398
I20260809 02:16:10.645852 23259 tablet_replica.cc:330] T b3f521c1d7344eb3a6bbf3f8e8ea485a P c6d25d1fc6294278a9c7ebdcda916e91: stopping tablet replica
I20260809 02:16:10.646134 23259 raft_consensus.cc:2229] T b3f521c1d7344eb3a6bbf3f8e8ea485a P c6d25d1fc6294278a9c7ebdcda916e91 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:16:10.646136 23258 raft_consensus.cc:2229] T b3f521c1d7344eb3a6bbf3f8e8ea485a P 8c50207057384e11affa724d35ca8768 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:16:10.646443 23258 raft_consensus.cc:2258] T b3f521c1d7344eb3a6bbf3f8e8ea485a P 8c50207057384e11affa724d35ca8768 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:16:10.646443 23259 raft_consensus.cc:2258] T b3f521c1d7344eb3a6bbf3f8e8ea485a P c6d25d1fc6294278a9c7ebdcda916e91 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:16:10.646852 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 22600
I20260809 02:16:10.649578 23261 tablet_replica.cc:330] T b3f521c1d7344eb3a6bbf3f8e8ea485a P 6f4a6204162d44b09eeac13b31e546e6: stopping tablet replica
I20260809 02:16:10.649873 23261 raft_consensus.cc:2229] T b3f521c1d7344eb3a6bbf3f8e8ea485a P 6f4a6204162d44b09eeac13b31e546e6 [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:16:10.650270 23261 raft_consensus.cc:2258] T b3f521c1d7344eb3a6bbf3f8e8ea485a P 6f4a6204162d44b09eeac13b31e546e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:16:10.651520 23259 ts_tablet_manager.cc:1905] T b3f521c1d7344eb3a6bbf3f8e8ea485a P c6d25d1fc6294278a9c7ebdcda916e91: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:16:10.654735 23259 ts_tablet_manager.cc:1918] T b3f521c1d7344eb3a6bbf3f8e8ea485a P c6d25d1fc6294278a9c7ebdcda916e91: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:16:10.654873 23259 log.cc:1196] T b3f521c1d7344eb3a6bbf3f8e8ea485a P c6d25d1fc6294278a9c7ebdcda916e91: Deleting WAL directory at /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241741788589-15662-0/minicluster-data/ts-2/wal/wals/b3f521c1d7344eb3a6bbf3f8e8ea485a
I20260809 02:16:10.655325 23259 ts_tablet_manager.cc:1939] T b3f521c1d7344eb3a6bbf3f8e8ea485a P c6d25d1fc6294278a9c7ebdcda916e91: Deleting consensus metadata
I20260809 02:16:10.657083 22484 catalog_manager.cc:4841] TS c6d25d1fc6294278a9c7ebdcda916e91 (127.15.75.131:39833): tablet b3f521c1d7344eb3a6bbf3f8e8ea485a (table test-table [id=97441ab009004a67806832e400ce42ad]) successfully deleted
I20260809 02:16:10.657855 23261 ts_tablet_manager.cc:1905] T b3f521c1d7344eb3a6bbf3f8e8ea485a P 6f4a6204162d44b09eeac13b31e546e6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:16:10.661484 23261 ts_tablet_manager.cc:1918] T b3f521c1d7344eb3a6bbf3f8e8ea485a P 6f4a6204162d44b09eeac13b31e546e6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:16:10.661598 23261 log.cc:1196] T b3f521c1d7344eb3a6bbf3f8e8ea485a P 6f4a6204162d44b09eeac13b31e546e6: Deleting WAL directory at /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241741788589-15662-0/minicluster-data/ts-1/wal/wals/b3f521c1d7344eb3a6bbf3f8e8ea485a
I20260809 02:16:10.661957 23261 ts_tablet_manager.cc:1939] T b3f521c1d7344eb3a6bbf3f8e8ea485a P 6f4a6204162d44b09eeac13b31e546e6: Deleting consensus metadata
I20260809 02:16:10.663285 22483 catalog_manager.cc:4841] TS 6f4a6204162d44b09eeac13b31e546e6 (127.15.75.130:41709): tablet b3f521c1d7344eb3a6bbf3f8e8ea485a (table test-table [id=97441ab009004a67806832e400ce42ad]) successfully deleted
W20260809 02:16:10.665256 22487 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.75.129:44151 (error 108)
I20260809 02:16:10.665401 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 22798
I20260809 02:16:10.674834 23219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:10.674531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.129:44151 (local address 127.0.0.1:43780)
0809 02:16:10.674672 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:10.674755 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.129:44151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260809 02:16:10.675010 22487 catalog_manager.cc:4584] TS 8c50207057384e11affa724d35ca8768 (127.15.75.129:44151): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b3f521c1d7344eb3a6bbf3f8e8ea485a: Network error: Client connection negotiation failed: client connection to 127.15.75.129:44151: connect: Connection refused (error 111)
I20260809 02:16:10.679879 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 23010
I20260809 02:16:10.688436 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 22455
2026-08-09T02:16:10Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1275 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260809 02:16:10.712424 15662 mini_oidc.cc:152] Starting JWKS server
I20260809 02:16:10.714258 15662 webserver.cc:460] Webserver started at https://127.0.0.1:36629/ using document root <none> and password file <none>
I20260809 02:16:10.714413 15662 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 02:16:10.714656 15662 webserver.cc:460] Webserver started at http://127.0.0.1:35227/ using document root <none> and password file <none>
2026-08-09T02:16:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:16:10Z Disabled control of system clock
I20260809 02:16:10.725095 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786241741788589-15662-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786241741788589-15662-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786241741788589-15662-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:45109
--webserver_interface=127.15.75.190
--webserver_port=0
--builtin_ntp_servers=127.15.75.148:37227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.75.190:45109
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36629/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786241741788589-15662-0/testchainca.cert with env {}
W20260809 02:16:10.841365 23269 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:16:10.841611 23269 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:16:10.841650 23269 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:16:10.845610 23269 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:16:10.845665 23269 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:16:10.845681 23269 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:16:10.845705 23269 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:16:10.845719 23269 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 02:16:10.845736 23269 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 02:16:10.845748 23269 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36629/jwks.json
I20260809 02:16:10.851007 23269 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:37227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786241741788589-15662-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786241741788589-15662-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.75.190:45109
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:45109
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36629/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.75.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786241741788589-15662-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23269
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786241741788589-15662-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:16:10.851948 23269 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:16:10.852919 23269 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 02:16:10.861531 23269 server_base.cc:1034] running on GCE node
W20260809 02:16:10.861478 23287 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:10.861485 23282 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:10.861485 23283 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:10.862373 23269 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:16:10.863508 23269 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:16:10.864777 23269 hybrid_clock.cc:648] HybridClock initialized: now 1786241770864743 us; error 41 us; skew 500 ppm
I20260809 02:16:10.866819 23269 webserver.cc:460] Webserver started at http://127.15.75.190:35365/ using document root <none> and password file <none>
I20260809 02:16:10.867443 23269 fs_manager.cc:359] Metadata directory not provided
I20260809 02:16:10.867544 23269 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:16:10.867798 23269 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:16:10.869990 23269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786241741788589-15662-0/minicluster-data/master-0/data/instance:
uuid: "1fba01b6a9374ea789188d27878ab69c"
format_stamp: "Formatted at 2026-08-09 02:16:10 on dist-test-slave-s11t"
server_key: "5c3adb57f0181bfafddaf6eb313d874c"
server_key_iv: "0d6a7f0353398e9ba8fd26938a217c27"
server_key_version: "encryptionkey@0"
I20260809 02:16:10.870594 23269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786241741788589-15662-0/minicluster-data/master-0/wal/instance:
uuid: "1fba01b6a9374ea789188d27878ab69c"
format_stamp: "Formatted at 2026-08-09 02:16:10 on dist-test-slave-s11t"
server_key: "5c3adb57f0181bfafddaf6eb313d874c"
server_key_iv: "0d6a7f0353398e9ba8fd26938a217c27"
server_key_version: "encryptionkey@0"
I20260809 02:16:10.874388 23269 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.001s
I20260809 02:16:10.876868 23296 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:10.878058 23269 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 02:16:10.878204 23269 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786241741788589-15662-0/minicluster-data/master-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786241741788589-15662-0/minicluster-data/master-0/wal
uuid: "1fba01b6a9374ea789188d27878ab69c"
format_stamp: "Formatted at 2026-08-09 02:16:10 on dist-test-slave-s11t"
server_key: "5c3adb57f0181bfafddaf6eb313d874c"
server_key_iv: "0d6a7f0353398e9ba8fd26938a217c27"
server_key_version: "encryptionkey@0"
I20260809 02:16:10.878328 23269 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786241741788589-15662-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786241741788589-15662-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786241741788589-15662-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:16:10.889873 23269 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:16:10.890511 23269 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:16:10.890705 23269 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:16:10.898120 23269 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.190:45109
I20260809 02:16:10.898128 23375 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.190:45109 every 8 connection(s)
I20260809 02:16:10.899096 23269 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
I20260809 02:16:10.900992 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 23269
I20260809 02:16:10.901130 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786241741788589-15662-0/minicluster-data/master-0/wal/instance
I20260809 02:16:10.901467 15662 external_mini_cluster.cc:1428] Setting key 7610f17dda3231d0d7f0dcc11b17ad66
I20260809 02:16:10.903242 23376 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:16:10.909041 23376 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1fba01b6a9374ea789188d27878ab69c: Bootstrap starting.
I20260809 02:16:10.910971 23384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:10.902846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51068 (local address 127.15.75.190:45109)
0809 02:16:10.903556 (+ 710us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:10.903572 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:10.903610 (+ 38us) server_negotiation.cc:407] Connection header received
0809 02:16:10.904285 (+ 675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:10.904302 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:10.904757 (+ 455us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:10.905103 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:10.905541 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:10.906596 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:10.907581 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:10.908086 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:10.908269 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:10.909228 (+ 959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:10.909234 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:10.909247 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:16:10.909646 (+ 399us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:10.909702 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:16:10.909723 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:10.909845 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:10.909985 (+ 140us) server_negotiation.cc:299] Negotiation successful
0809 02:16:10.910255 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":149,"threads_started":1}
I20260809 02:16:10.911504 23376 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1fba01b6a9374ea789188d27878ab69c: Neither blocks nor log segments found. Creating new log.
I20260809 02:16:10.912424 23376 log.cc:824] T 00000000000000000000000000000000 P 1fba01b6a9374ea789188d27878ab69c: Log is configured to *not* fsync() on all Append() calls
I20260809 02:16:10.914003 15662 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:16:10.914562 23376 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1fba01b6a9374ea789188d27878ab69c: No bootstrap required, opened a new log
I20260809 02:16:10.918309 23376 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1fba01b6a9374ea789188d27878ab69c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fba01b6a9374ea789188d27878ab69c" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 45109 } }
I20260809 02:16:10.918562 23376 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1fba01b6a9374ea789188d27878ab69c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:16:10.918648 23376 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1fba01b6a9374ea789188d27878ab69c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fba01b6a9374ea789188d27878ab69c, State: Initialized, Role: FOLLOWER
I20260809 02:16:10.919090 23376 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1fba01b6a9374ea789188d27878ab69c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fba01b6a9374ea789188d27878ab69c" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 45109 } }
I20260809 02:16:10.919242 23376 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1fba01b6a9374ea789188d27878ab69c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:16:10.919338 23376 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1fba01b6a9374ea789188d27878ab69c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:16:10.919456 23376 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1fba01b6a9374ea789188d27878ab69c [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:16:10.920434 23376 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1fba01b6a9374ea789188d27878ab69c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fba01b6a9374ea789188d27878ab69c" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 45109 } }
I20260809 02:16:10.920825 23376 leader_election.cc:302] T 00000000000000000000000000000000 P 1fba01b6a9374ea789188d27878ab69c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1fba01b6a9374ea789188d27878ab69c; no voters:
I20260809 02:16:10.921162 23376 leader_election.cc:288] T 00000000000000000000000000000000 P 1fba01b6a9374ea789188d27878ab69c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:16:10.921285 23386 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1fba01b6a9374ea789188d27878ab69c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:16:10.921530 23386 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1fba01b6a9374ea789188d27878ab69c [term 1 LEADER]: Becoming Leader. State: Replica: 1fba01b6a9374ea789188d27878ab69c, State: Running, Role: LEADER
I20260809 02:16:10.921873 23386 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1fba01b6a9374ea789188d27878ab69c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fba01b6a9374ea789188d27878ab69c" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 45109 } }
I20260809 02:16:10.922515 23376 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1fba01b6a9374ea789188d27878ab69c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:16:10.923895 23388 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fba01b6a9374ea789188d27878ab69c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1fba01b6a9374ea789188d27878ab69c. Latest consensus state: current_term: 1 leader_uuid: "1fba01b6a9374ea789188d27878ab69c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fba01b6a9374ea789188d27878ab69c" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 45109 } } }
I20260809 02:16:10.924073 23388 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fba01b6a9374ea789188d27878ab69c [sys.catalog]: This master's current role is: LEADER
I20260809 02:16:10.924357 23387 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fba01b6a9374ea789188d27878ab69c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1fba01b6a9374ea789188d27878ab69c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fba01b6a9374ea789188d27878ab69c" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 45109 } } }
I20260809 02:16:10.924472 23387 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fba01b6a9374ea789188d27878ab69c [sys.catalog]: This master's current role is: LEADER
I20260809 02:16:10.924990 23398 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:16:10.928400 23398 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:16:10.934697 23398 catalog_manager.cc:1347] Generated new cluster ID: d4a8327eceb549ffa3afc2f7cd9df695
I20260809 02:16:10.934789 23398 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:16:10.946449 23398 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:16:10.947419 23398 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:16:10.956827 23398 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1fba01b6a9374ea789188d27878ab69c: Generated new TSK 0
I20260809 02:16:10.957458 23398 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 02:16:10.990302 23428 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260809 02:16:10.990499 23428 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 02:16:10.991125 23384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:10.974270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51072 (local address 127.15.75.190:45109)
0809 02:16:10.974422 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:10.974427 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:10.974618 (+ 191us) server_negotiation.cc:407] Connection header received
0809 02:16:10.974849 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:10.974857 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:10.974905 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:10.975003 (+ 98us) server_negotiation.cc:226] Negotiated authn=JWT
0809 02:16:10.975281 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:10.975817 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:10.976729 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:10.976947 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:10.977058 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:10.977414 (+ 356us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 02:16:10.990694 (+ 13280us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36629/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 02:16:10.990974 (+ 280us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51072: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36629/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":68}
W20260809 02:16:10.991200 23384 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51072: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36629/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260809 02:16:10.992376 23427 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:16:10.974177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:45109 (local address 127.0.0.1:51072)
0809 02:16:10.974536 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:10.974548 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:10.974693 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:10.975034 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:10.975037 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:10.975045 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0809 02:16:10.975174 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:10.975180 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:10.975937 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:10.975941 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:10.976583 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:10.976591 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:10.977067 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:10.977071 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:10.977173 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:10.977216 (+ 43us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 02:16:10.990923 (+ 13707us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36629/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 02:16:10.991179 (+ 256us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.15.75.190:45109: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36629/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":121,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.15.75.190:45109: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36629/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260809 02:16:10.993125 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 23269
2026-08-09T02:16:11Z chronyd exiting
I20260809 02:16:11.035470 15662 test_util.cc:175] -----------------------------------------------
I20260809 02:16:11.035568 15662 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786241741788589-15662-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (324 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260809 02:16:11.036644 15662 mini_oidc.cc:152] Starting JWKS server
I20260809 02:16:11.037534 15662 webserver.cc:460] Webserver started at https://127.0.0.1:36267/ using document root <none> and password file <none>
I20260809 02:16:11.037657 15662 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 02:16:11.037917 15662 webserver.cc:460] Webserver started at http://127.0.0.1:34823/ using document root <none> and password file <none>
2026-08-09T02:16:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:16:11Z Disabled control of system clock
I20260809 02:16:11.047850 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786241741788589-15662-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786241741788589-15662-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786241741788589-15662-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:34425
--webserver_interface=127.15.75.190
--webserver_port=0
--builtin_ntp_servers=127.15.75.148:41233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.75.190:34425
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36267/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786241741788589-15662-0/testchainca.cert with env {}
W20260809 02:16:11.159644 23436 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:16:11.159884 23436 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:16:11.159924 23436 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:16:11.163885 23436 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:16:11.163941 23436 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:16:11.163957 23436 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:16:11.163973 23436 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:16:11.163988 23436 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 02:16:11.164004 23436 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 02:16:11.164021 23436 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36267/jwks.json
I20260809 02:16:11.169164 23436 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:41233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786241741788589-15662-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786241741788589-15662-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.75.190:34425
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:34425
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36267/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.75.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786241741788589-15662-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23436
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786241741788589-15662-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:16:11.170063 23436 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:16:11.170962 23436 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:16:11.179236 23453 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:11.179224 23451 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:11.179229 23448 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:11.179581 23436 server_base.cc:1034] running on GCE node
I20260809 02:16:11.180131 23436 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:16:11.181206 23436 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:16:11.182394 23436 hybrid_clock.cc:648] HybridClock initialized: now 1786241771182377 us; error 26 us; skew 500 ppm
I20260809 02:16:11.184146 23436 webserver.cc:460] Webserver started at http://127.15.75.190:33825/ using document root <none> and password file <none>
I20260809 02:16:11.184757 23436 fs_manager.cc:359] Metadata directory not provided
I20260809 02:16:11.184847 23436 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:16:11.185072 23436 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:16:11.187106 23436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786241741788589-15662-0/minicluster-data/master-0/data/instance:
uuid: "a488ed970fa54dbf9ee80d98fba6d91f"
format_stamp: "Formatted at 2026-08-09 02:16:11 on dist-test-slave-s11t"
server_key: "d042a779556732bbdfa9a46a392b635e"
server_key_iv: "eac9d9c325a3514d4d454b57a17d00d7"
server_key_version: "encryptionkey@0"
I20260809 02:16:11.187664 23436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786241741788589-15662-0/minicluster-data/master-0/wal/instance:
uuid: "a488ed970fa54dbf9ee80d98fba6d91f"
format_stamp: "Formatted at 2026-08-09 02:16:11 on dist-test-slave-s11t"
server_key: "d042a779556732bbdfa9a46a392b635e"
server_key_iv: "eac9d9c325a3514d4d454b57a17d00d7"
server_key_version: "encryptionkey@0"
I20260809 02:16:11.191152 23436 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260809 02:16:11.193459 23460 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:11.194638 23436 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 02:16:11.194783 23436 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786241741788589-15662-0/minicluster-data/master-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786241741788589-15662-0/minicluster-data/master-0/wal
uuid: "a488ed970fa54dbf9ee80d98fba6d91f"
format_stamp: "Formatted at 2026-08-09 02:16:11 on dist-test-slave-s11t"
server_key: "d042a779556732bbdfa9a46a392b635e"
server_key_iv: "eac9d9c325a3514d4d454b57a17d00d7"
server_key_version: "encryptionkey@0"
I20260809 02:16:11.194887 23436 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786241741788589-15662-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786241741788589-15662-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786241741788589-15662-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:16:11.206068 23436 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:16:11.206638 23436 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:16:11.206830 23436 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:16:11.213455 23436 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.190:34425
I20260809 02:16:11.213488 23545 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.190:34425 every 8 connection(s)
I20260809 02:16:11.214296 23436 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
I20260809 02:16:11.217411 23547 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:16:11.222522 23547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a488ed970fa54dbf9ee80d98fba6d91f: Bootstrap starting.
I20260809 02:16:11.223291 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 23436
I20260809 02:16:11.223405 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786241741788589-15662-0/minicluster-data/master-0/wal/instance
I20260809 02:16:11.223754 15662 external_mini_cluster.cc:1428] Setting key fa688d537f4d1891f5838e4013014974
I20260809 02:16:11.225415 23547 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a488ed970fa54dbf9ee80d98fba6d91f: Neither blocks nor log segments found. Creating new log.
I20260809 02:16:11.226133 23547 log.cc:824] T 00000000000000000000000000000000 P a488ed970fa54dbf9ee80d98fba6d91f: Log is configured to *not* fsync() on all Append() calls
I20260809 02:16:11.227861 23547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a488ed970fa54dbf9ee80d98fba6d91f: No bootstrap required, opened a new log
I20260809 02:16:11.231134 23547 raft_consensus.cc:348] T 00000000000000000000000000000000 P a488ed970fa54dbf9ee80d98fba6d91f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a488ed970fa54dbf9ee80d98fba6d91f" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 34425 } }
I20260809 02:16:11.231354 23547 raft_consensus.cc:374] T 00000000000000000000000000000000 P a488ed970fa54dbf9ee80d98fba6d91f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:16:11.231427 23547 raft_consensus.cc:729] T 00000000000000000000000000000000 P a488ed970fa54dbf9ee80d98fba6d91f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a488ed970fa54dbf9ee80d98fba6d91f, State: Initialized, Role: FOLLOWER
I20260809 02:16:11.231843 23547 consensus_queue.cc:260] T 00000000000000000000000000000000 P a488ed970fa54dbf9ee80d98fba6d91f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a488ed970fa54dbf9ee80d98fba6d91f" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 34425 } }
I20260809 02:16:11.231987 23547 raft_consensus.cc:388] T 00000000000000000000000000000000 P a488ed970fa54dbf9ee80d98fba6d91f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:16:11.232077 23547 raft_consensus.cc:482] T 00000000000000000000000000000000 P a488ed970fa54dbf9ee80d98fba6d91f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:16:11.232196 23547 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a488ed970fa54dbf9ee80d98fba6d91f [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:16:11.232605 23551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:11.224918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55674 (local address 127.15.75.190:34425)
0809 02:16:11.225497 (+ 579us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:11.225528 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:11.225573 (+ 45us) server_negotiation.cc:407] Connection header received
0809 02:16:11.226179 (+ 606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:11.226195 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:11.226605 (+ 410us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:11.226923 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:11.227316 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:11.228297 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:11.229254 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:11.229715 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:11.229893 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:11.230693 (+ 800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:11.230698 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:11.230710 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:16:11.231074 (+ 364us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:11.231132 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:16:11.231151 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:11.231255 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:11.231370 (+ 115us) server_negotiation.cc:299] Negotiation successful
0809 02:16:11.231759 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":253,"thread_start_us":131,"threads_started":1}
I20260809 02:16:11.233192 23547 raft_consensus.cc:504] T 00000000000000000000000000000000 P a488ed970fa54dbf9ee80d98fba6d91f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a488ed970fa54dbf9ee80d98fba6d91f" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 34425 } }
I20260809 02:16:11.233525 23547 leader_election.cc:302] T 00000000000000000000000000000000 P a488ed970fa54dbf9ee80d98fba6d91f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a488ed970fa54dbf9ee80d98fba6d91f; no voters:
I20260809 02:16:11.233853 23547 leader_election.cc:288] T 00000000000000000000000000000000 P a488ed970fa54dbf9ee80d98fba6d91f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:16:11.233915 23553 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a488ed970fa54dbf9ee80d98fba6d91f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:16:11.234124 23553 raft_consensus.cc:686] T 00000000000000000000000000000000 P a488ed970fa54dbf9ee80d98fba6d91f [term 1 LEADER]: Becoming Leader. State: Replica: a488ed970fa54dbf9ee80d98fba6d91f, State: Running, Role: LEADER
I20260809 02:16:11.234408 23553 consensus_queue.cc:237] T 00000000000000000000000000000000 P a488ed970fa54dbf9ee80d98fba6d91f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a488ed970fa54dbf9ee80d98fba6d91f" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 34425 } }
I20260809 02:16:11.235059 23547 sys_catalog.cc:564] T 00000000000000000000000000000000 P a488ed970fa54dbf9ee80d98fba6d91f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:16:11.235478 23553 sys_catalog.cc:455] T 00000000000000000000000000000000 P a488ed970fa54dbf9ee80d98fba6d91f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a488ed970fa54dbf9ee80d98fba6d91f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a488ed970fa54dbf9ee80d98fba6d91f" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 34425 } } }
I20260809 02:16:11.235591 23553 sys_catalog.cc:458] T 00000000000000000000000000000000 P a488ed970fa54dbf9ee80d98fba6d91f [sys.catalog]: This master's current role is: LEADER
I20260809 02:16:11.235615 15662 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:16:11.236032 23554 sys_catalog.cc:455] T 00000000000000000000000000000000 P a488ed970fa54dbf9ee80d98fba6d91f [sys.catalog]: SysCatalogTable state changed. Reason: New leader a488ed970fa54dbf9ee80d98fba6d91f. Latest consensus state: current_term: 1 leader_uuid: "a488ed970fa54dbf9ee80d98fba6d91f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a488ed970fa54dbf9ee80d98fba6d91f" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 34425 } } }
I20260809 02:16:11.236181 23554 sys_catalog.cc:458] T 00000000000000000000000000000000 P a488ed970fa54dbf9ee80d98fba6d91f [sys.catalog]: This master's current role is: LEADER
I20260809 02:16:11.237367 23557 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:16:11.240085 23557 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:16:11.244853 23557 catalog_manager.cc:1347] Generated new cluster ID: 0b81084eb26b4367ba3a1e6f6ef6cb79
I20260809 02:16:11.244913 23557 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:16:11.253441 23557 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:16:11.254541 23557 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:16:11.264520 23557 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a488ed970fa54dbf9ee80d98fba6d91f: Generated new TSK 0
I20260809 02:16:11.265369 23557 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 02:16:11.291556 23593 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260809 02:16:11.291643 23593 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 02:16:11.292351 23551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:11.277040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55678 (local address 127.15.75.190:34425)
0809 02:16:11.277179 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:11.277184 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:11.277379 (+ 195us) server_negotiation.cc:407] Connection header received
0809 02:16:11.277558 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:11.277561 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:11.277605 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:11.277713 (+ 108us) server_negotiation.cc:226] Negotiated authn=JWT
0809 02:16:11.277954 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:11.278500 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:11.279376 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:11.279608 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:11.279721 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:11.280027 (+ 306us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 02:16:11.291926 (+ 11899us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36267/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 02:16:11.292192 (+ 266us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55678: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36267/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":47}
W20260809 02:16:11.292407 23591 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:16:11.276999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:34425 (local address 127.0.0.1:55678)
0809 02:16:11.277287 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:11.277299 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:11.277426 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:11.277718 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:11.277721 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:11.277729 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0809 02:16:11.277841 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:11.277847 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:11.278614 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:11.278619 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:11.279256 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:11.279263 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:11.279739 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:11.279744 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:11.279832 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:11.279860 (+ 28us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 02:16:11.292084 (+ 12224us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36267/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 02:16:11.292240 (+ 156us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.15.75.190:34425: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36267/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":105,"threads_started":1}
W20260809 02:16:11.292423 23551 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55678: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36267/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
I20260809 02:16:11.293066 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 23436
2026-08-09T02:16:11Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (303 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260809 02:16:11.340054 15662 mini_oidc.cc:152] Starting JWKS server
I20260809 02:16:11.341017 15662 webserver.cc:460] Webserver started at https://127.0.0.1:37035/ using document root <none> and password file <none>
I20260809 02:16:11.341145 15662 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 02:16:11.341419 15662 webserver.cc:460] Webserver started at http://127.0.0.1:36985/ using document root <none> and password file <none>
2026-08-09T02:16:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:16:11Z Disabled control of system clock
I20260809 02:16:11.351670 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786241741788589-15662-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786241741788589-15662-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786241741788589-15662-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:35091
--webserver_interface=127.15.75.190
--webserver_port=0
--builtin_ntp_servers=127.15.75.148:38459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.75.190:35091
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37035/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 02:16:11.461433 23599 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:16:11.461673 23599 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:16:11.461709 23599 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:16:11.465679 23599 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:16:11.465739 23599 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:16:11.465758 23599 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:16:11.465775 23599 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:16:11.465799 23599 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 02:16:11.465821 23599 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 02:16:11.465840 23599 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:37035/jwks.json
I20260809 02:16:11.470512 23599 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:38459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786241741788589-15662-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786241741788589-15662-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.75.190:35091
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:35091
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37035/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.75.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23599
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786241741788589-15662-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:16:11.471493 23599 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:16:11.472826 23599 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:16:11.481154 23613 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:11.481130 23614 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:11.481154 23617 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:11.481387 23599 server_base.cc:1034] running on GCE node
I20260809 02:16:11.482008 23599 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:16:11.483093 23599 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:16:11.484288 23599 hybrid_clock.cc:648] HybridClock initialized: now 1786241771484278 us; error 30 us; skew 500 ppm
I20260809 02:16:11.486145 23599 webserver.cc:460] Webserver started at http://127.15.75.190:38621/ using document root <none> and password file <none>
I20260809 02:16:11.486696 23599 fs_manager.cc:359] Metadata directory not provided
I20260809 02:16:11.486791 23599 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:16:11.487015 23599 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:16:11.489069 23599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786241741788589-15662-0/minicluster-data/master-0/data/instance:
uuid: "a3710a723fd649ab86aa61151c220b97"
format_stamp: "Formatted at 2026-08-09 02:16:11 on dist-test-slave-s11t"
server_key: "1c86dac93d931572e41f4329aa4d3870"
server_key_iv: "7f83f4ae01beb315541d461ea26106d4"
server_key_version: "encryptionkey@0"
I20260809 02:16:11.489617 23599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786241741788589-15662-0/minicluster-data/master-0/wal/instance:
uuid: "a3710a723fd649ab86aa61151c220b97"
format_stamp: "Formatted at 2026-08-09 02:16:11 on dist-test-slave-s11t"
server_key: "1c86dac93d931572e41f4329aa4d3870"
server_key_iv: "7f83f4ae01beb315541d461ea26106d4"
server_key_version: "encryptionkey@0"
I20260809 02:16:11.493152 23599 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 02:16:11.495424 23632 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:11.496856 23599 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 02:16:11.496986 23599 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786241741788589-15662-0/minicluster-data/master-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786241741788589-15662-0/minicluster-data/master-0/wal
uuid: "a3710a723fd649ab86aa61151c220b97"
format_stamp: "Formatted at 2026-08-09 02:16:11 on dist-test-slave-s11t"
server_key: "1c86dac93d931572e41f4329aa4d3870"
server_key_iv: "7f83f4ae01beb315541d461ea26106d4"
server_key_version: "encryptionkey@0"
I20260809 02:16:11.497094 23599 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786241741788589-15662-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786241741788589-15662-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786241741788589-15662-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:16:11.509390 23599 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:16:11.509989 23599 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:16:11.510170 23599 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:16:11.517362 23599 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.190:35091
I20260809 02:16:11.517372 23706 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.190:35091 every 8 connection(s)
I20260809 02:16:11.518242 23599 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
I20260809 02:16:11.521189 23708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:16:11.526538 23708 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3710a723fd649ab86aa61151c220b97: Bootstrap starting.
I20260809 02:16:11.527102 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 23599
I20260809 02:16:11.527238 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786241741788589-15662-0/minicluster-data/master-0/wal/instance
I20260809 02:16:11.527498 15662 external_mini_cluster.cc:1428] Setting key 36acf0e317b93f58ce3569038067125a
I20260809 02:16:11.529241 23708 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a3710a723fd649ab86aa61151c220b97: Neither blocks nor log segments found. Creating new log.
I20260809 02:16:11.530119 23708 log.cc:824] T 00000000000000000000000000000000 P a3710a723fd649ab86aa61151c220b97: Log is configured to *not* fsync() on all Append() calls
I20260809 02:16:11.532426 23708 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3710a723fd649ab86aa61151c220b97: No bootstrap required, opened a new log
I20260809 02:16:11.536849 23708 raft_consensus.cc:348] T 00000000000000000000000000000000 P a3710a723fd649ab86aa61151c220b97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3710a723fd649ab86aa61151c220b97" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 35091 } }
I20260809 02:16:11.537111 23708 raft_consensus.cc:374] T 00000000000000000000000000000000 P a3710a723fd649ab86aa61151c220b97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:16:11.537166 23708 raft_consensus.cc:729] T 00000000000000000000000000000000 P a3710a723fd649ab86aa61151c220b97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3710a723fd649ab86aa61151c220b97, State: Initialized, Role: FOLLOWER
I20260809 02:16:11.537609 23708 consensus_queue.cc:260] T 00000000000000000000000000000000 P a3710a723fd649ab86aa61151c220b97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3710a723fd649ab86aa61151c220b97" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 35091 } }
I20260809 02:16:11.537767 23708 raft_consensus.cc:388] T 00000000000000000000000000000000 P a3710a723fd649ab86aa61151c220b97 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:16:11.537834 23708 raft_consensus.cc:482] T 00000000000000000000000000000000 P a3710a723fd649ab86aa61151c220b97 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:16:11.537886 23712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:11.528797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49746 (local address 127.15.75.190:35091)
0809 02:16:11.529519 (+ 722us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:11.529539 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:11.529581 (+ 42us) server_negotiation.cc:407] Connection header received
0809 02:16:11.530441 (+ 860us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:11.530467 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:11.530844 (+ 377us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:11.531169 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:11.531593 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:11.533095 (+ 1502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:11.534136 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:11.534828 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:11.534999 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:11.535772 (+ 773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:11.535781 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:11.535801 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:16:11.536323 (+ 522us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:11.536406 (+ 83us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:16:11.536433 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:11.536530 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:11.536722 (+ 192us) server_negotiation.cc:299] Negotiation successful
0809 02:16:11.536978 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":109,"threads_started":1}
I20260809 02:16:11.537943 23708 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a3710a723fd649ab86aa61151c220b97 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:16:11.538765 23708 raft_consensus.cc:504] T 00000000000000000000000000000000 P a3710a723fd649ab86aa61151c220b97 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3710a723fd649ab86aa61151c220b97" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 35091 } }
I20260809 02:16:11.539089 23708 leader_election.cc:302] T 00000000000000000000000000000000 P a3710a723fd649ab86aa61151c220b97 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a3710a723fd649ab86aa61151c220b97; no voters:
I20260809 02:16:11.539402 23708 leader_election.cc:288] T 00000000000000000000000000000000 P a3710a723fd649ab86aa61151c220b97 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:16:11.539479 23716 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a3710a723fd649ab86aa61151c220b97 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:16:11.539673 23716 raft_consensus.cc:686] T 00000000000000000000000000000000 P a3710a723fd649ab86aa61151c220b97 [term 1 LEADER]: Becoming Leader. State: Replica: a3710a723fd649ab86aa61151c220b97, State: Running, Role: LEADER
I20260809 02:16:11.539949 23716 consensus_queue.cc:237] T 00000000000000000000000000000000 P a3710a723fd649ab86aa61151c220b97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3710a723fd649ab86aa61151c220b97" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 35091 } }
I20260809 02:16:11.540630 23708 sys_catalog.cc:564] T 00000000000000000000000000000000 P a3710a723fd649ab86aa61151c220b97 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:16:11.540705 15662 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:16:11.541567 23718 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3710a723fd649ab86aa61151c220b97 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a3710a723fd649ab86aa61151c220b97. Latest consensus state: current_term: 1 leader_uuid: "a3710a723fd649ab86aa61151c220b97" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3710a723fd649ab86aa61151c220b97" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 35091 } } }
I20260809 02:16:11.541728 23718 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3710a723fd649ab86aa61151c220b97 [sys.catalog]: This master's current role is: LEADER
I20260809 02:16:11.542049 23719 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3710a723fd649ab86aa61151c220b97 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a3710a723fd649ab86aa61151c220b97" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3710a723fd649ab86aa61151c220b97" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 35091 } } }
I20260809 02:16:11.542145 23719 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3710a723fd649ab86aa61151c220b97 [sys.catalog]: This master's current role is: LEADER
W20260809 02:16:11.545356 23740 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a3710a723fd649ab86aa61151c220b97: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:16:11.545445 23740 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:16:11.545524 23733 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:16:11.546221 23733 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:16:11.551994 23733 catalog_manager.cc:1347] Generated new cluster ID: 883c5e0b65704680a71070b526dc8b04
I20260809 02:16:11.552057 23733 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:16:11.574990 23733 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:16:11.576243 23733 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:16:11.581283 23733 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a3710a723fd649ab86aa61151c220b97: Generated new TSK 0
I20260809 02:16:11.582000 23733 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:16:11.597909 23712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:11.592818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49762 (local address 127.15.75.190:35091)
0809 02:16:11.592952 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:11.592957 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:11.593162 (+ 205us) server_negotiation.cc:407] Connection header received
0809 02:16:11.593366 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:11.593369 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:11.593423 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:11.593517 (+ 94us) server_negotiation.cc:226] Negotiated authn=JWT
0809 02:16:11.593770 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:11.594300 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:11.595555 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:11.595790 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:11.595871 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:11.596184 (+ 313us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 02:16:11.597565 (+ 1381us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37035/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 02:16:11.597788 (+ 223us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49762: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37035/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":50}
W20260809 02:16:11.597970 23712 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49762: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37035/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260809 02:16:11.599776 23750 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:16:11.592770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:35091 (local address 127.0.0.1:49762)
0809 02:16:11.593069 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:11.593084 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:11.593198 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:11.593536 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:11.593540 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:11.593548 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0809 02:16:11.593662 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:11.593668 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:11.594413 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:11.594417 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:11.595423 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:11.595435 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:11.595903 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:11.595908 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:11.596000 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:11.596028 (+ 28us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 02:16:11.597718 (+ 1690us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37035/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 02:16:11.599595 (+ 1877us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.15.75.190:35091: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37035/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":213,"mutex_wait_us":1672,"thread_start_us":118,"threads_started":1}
I20260809 02:16:11.600317 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 23599
E20260809 02:16:11.616533 23752 webserver.cc:533] Webserver: sslize failed:
E20260809 02:16:11.616604 23752 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-09T02:16:11Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (309 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-09T02:16:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:16:11Z Disabled control of system clock
I20260809 02:16:11.659060 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786241741788589-15662-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786241741788589-15662-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786241741788589-15662-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:38627
--webserver_interface=127.15.75.190
--webserver_port=0
--builtin_ntp_servers=127.15.75.148:43689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.75.190:38627
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786241741788589-15662-0/testchainca.cert with env {}
W20260809 02:16:11.769186 23756 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:16:11.769424 23756 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:16:11.769462 23756 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:16:11.773325 23756 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:16:11.773372 23756 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:16:11.773388 23756 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:16:11.773404 23756 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:16:11.773419 23756 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 02:16:11.773442 23756 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 02:16:11.773458 23756 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260809 02:16:11.778506 23756 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:43689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786241741788589-15662-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786241741788589-15662-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.75.190:38627
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:38627
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.75.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786241741788589-15662-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23756
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786241741788589-15662-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:16:11.779390 23756 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:16:11.780267 23756 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:16:11.788551 23769 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:11.788550 23770 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:11.788570 23774 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:11.788959 23756 server_base.cc:1034] running on GCE node
I20260809 02:16:11.789547 23756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:16:11.790577 23756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:16:11.791767 23756 hybrid_clock.cc:648] HybridClock initialized: now 1786241771791746 us; error 31 us; skew 500 ppm
I20260809 02:16:11.793573 23756 webserver.cc:460] Webserver started at http://127.15.75.190:37831/ using document root <none> and password file <none>
I20260809 02:16:11.794188 23756 fs_manager.cc:359] Metadata directory not provided
I20260809 02:16:11.794279 23756 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:16:11.794497 23756 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:16:11.796479 23756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786241741788589-15662-0/minicluster-data/master-0/data/instance:
uuid: "4830018fd1cc428aaf037c3976196486"
format_stamp: "Formatted at 2026-08-09 02:16:11 on dist-test-slave-s11t"
server_key: "ed599c82042a135e065dadba4284456c"
server_key_iv: "67f87b7cbd44004a64a3404fd16b3d41"
server_key_version: "encryptionkey@0"
I20260809 02:16:11.797093 23756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786241741788589-15662-0/minicluster-data/master-0/wal/instance:
uuid: "4830018fd1cc428aaf037c3976196486"
format_stamp: "Formatted at 2026-08-09 02:16:11 on dist-test-slave-s11t"
server_key: "ed599c82042a135e065dadba4284456c"
server_key_iv: "67f87b7cbd44004a64a3404fd16b3d41"
server_key_version: "encryptionkey@0"
I20260809 02:16:11.800773 23756 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 02:16:11.803123 23782 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:11.804366 23756 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 02:16:11.804529 23756 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786241741788589-15662-0/minicluster-data/master-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786241741788589-15662-0/minicluster-data/master-0/wal
uuid: "4830018fd1cc428aaf037c3976196486"
format_stamp: "Formatted at 2026-08-09 02:16:11 on dist-test-slave-s11t"
server_key: "ed599c82042a135e065dadba4284456c"
server_key_iv: "67f87b7cbd44004a64a3404fd16b3d41"
server_key_version: "encryptionkey@0"
I20260809 02:16:11.804642 23756 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786241741788589-15662-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786241741788589-15662-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786241741788589-15662-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:16:11.818276 23756 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:16:11.818882 23756 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:16:11.819092 23756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:16:11.825937 23756 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.190:38627
I20260809 02:16:11.825954 23860 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.190:38627 every 8 connection(s)
I20260809 02:16:11.826833 23756 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
I20260809 02:16:11.829567 23861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:16:11.834764 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 23756
I20260809 02:16:11.834883 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786241741788589-15662-0/minicluster-data/master-0/wal/instance
I20260809 02:16:11.835022 23861 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4830018fd1cc428aaf037c3976196486: Bootstrap starting.
I20260809 02:16:11.835196 15662 external_mini_cluster.cc:1428] Setting key c773b6a82e0039742c77879068ae6f46
I20260809 02:16:11.837582 23861 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4830018fd1cc428aaf037c3976196486: Neither blocks nor log segments found. Creating new log.
I20260809 02:16:11.838299 23861 log.cc:824] T 00000000000000000000000000000000 P 4830018fd1cc428aaf037c3976196486: Log is configured to *not* fsync() on all Append() calls
I20260809 02:16:11.840202 23861 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4830018fd1cc428aaf037c3976196486: No bootstrap required, opened a new log
I20260809 02:16:11.844250 23861 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4830018fd1cc428aaf037c3976196486 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4830018fd1cc428aaf037c3976196486" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 38627 } }
I20260809 02:16:11.844389 23865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:11.836515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45736 (local address 127.15.75.190:38627)
0809 02:16:11.837165 (+ 650us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:11.837180 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:11.837214 (+ 34us) server_negotiation.cc:407] Connection header received
0809 02:16:11.837887 (+ 673us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:11.837906 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:11.838276 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:11.838585 (+ 309us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:11.839046 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:11.840040 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:11.841250 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:11.841709 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:11.841854 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:11.842669 (+ 815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:11.842674 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:11.842697 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:16:11.843087 (+ 390us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:11.843140 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:16:11.843161 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:11.843237 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:11.843417 (+ 180us) server_negotiation.cc:299] Negotiation successful
0809 02:16:11.843677 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":157,"threads_started":1}
I20260809 02:16:11.844460 23861 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4830018fd1cc428aaf037c3976196486 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:16:11.844522 23861 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4830018fd1cc428aaf037c3976196486 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4830018fd1cc428aaf037c3976196486, State: Initialized, Role: FOLLOWER
I20260809 02:16:11.845002 23861 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4830018fd1cc428aaf037c3976196486 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4830018fd1cc428aaf037c3976196486" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 38627 } }
I20260809 02:16:11.845129 23861 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4830018fd1cc428aaf037c3976196486 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:16:11.845216 23861 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4830018fd1cc428aaf037c3976196486 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:16:11.845332 23861 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4830018fd1cc428aaf037c3976196486 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:16:11.846362 23861 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4830018fd1cc428aaf037c3976196486 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4830018fd1cc428aaf037c3976196486" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 38627 } }
I20260809 02:16:11.846717 23861 leader_election.cc:302] T 00000000000000000000000000000000 P 4830018fd1cc428aaf037c3976196486 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4830018fd1cc428aaf037c3976196486; no voters:
I20260809 02:16:11.847064 23861 leader_election.cc:288] T 00000000000000000000000000000000 P 4830018fd1cc428aaf037c3976196486 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:16:11.847213 23867 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4830018fd1cc428aaf037c3976196486 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:16:11.847325 15662 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:16:11.847470 23867 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4830018fd1cc428aaf037c3976196486 [term 1 LEADER]: Becoming Leader. State: Replica: 4830018fd1cc428aaf037c3976196486, State: Running, Role: LEADER
I20260809 02:16:11.847782 23867 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4830018fd1cc428aaf037c3976196486 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4830018fd1cc428aaf037c3976196486" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 38627 } }
I20260809 02:16:11.848295 23861 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4830018fd1cc428aaf037c3976196486 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:16:11.849401 23870 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4830018fd1cc428aaf037c3976196486 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4830018fd1cc428aaf037c3976196486" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4830018fd1cc428aaf037c3976196486" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 38627 } } }
I20260809 02:16:11.849535 23870 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4830018fd1cc428aaf037c3976196486 [sys.catalog]: This master's current role is: LEADER
I20260809 02:16:11.849768 23868 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4830018fd1cc428aaf037c3976196486 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4830018fd1cc428aaf037c3976196486. Latest consensus state: current_term: 1 leader_uuid: "4830018fd1cc428aaf037c3976196486" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4830018fd1cc428aaf037c3976196486" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 38627 } } }
I20260809 02:16:11.849874 23868 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4830018fd1cc428aaf037c3976196486 [sys.catalog]: This master's current role is: LEADER
I20260809 02:16:11.850029 23883 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:16:11.853173 23883 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:16:11.858528 23883 catalog_manager.cc:1347] Generated new cluster ID: e9c5dbf5e5474000989c25847ce5e664
I20260809 02:16:11.858608 23883 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:16:11.887920 23883 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:16:11.888895 23883 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:16:11.905372 23883 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4830018fd1cc428aaf037c3976196486: Generated new TSK 0
I20260809 02:16:11.906296 23883 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:16:11.984562 23865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:11.918421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45742 (local address 127.15.75.190:38627)
0809 02:16:11.918558 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:11.918563 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:11.918757 (+ 194us) server_negotiation.cc:407] Connection header received
0809 02:16:11.918913 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:11.918915 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:11.918961 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:11.919048 (+ 87us) server_negotiation.cc:226] Negotiated authn=JWT
0809 02:16:11.919323 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:11.919853 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:11.921107 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:11.921334 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:11.921414 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:11.921736 (+ 322us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 02:16:11.984108 (+ 62372us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 02:16:11.984379 (+ 271us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45742: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":52}
W20260809 02:16:11.984653 23865 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45742: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260809 02:16:11.984834 23912 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:16:11.918363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:38627 (local address 127.0.0.1:45742)
0809 02:16:11.918666 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:11.918682 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:11.918799 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:11.919068 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:11.919071 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:11.919080 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0809 02:16:11.919201 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:11.919208 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:11.919957 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:11.919964 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:11.920985 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:11.920996 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:11.921455 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:11.921459 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:11.921558 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:11.921587 (+ 29us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 02:16:11.984380 (+ 62793us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 02:16:11.984617 (+ 237us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.15.75.190:38627: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":129,"threads_started":1}
I20260809 02:16:11.985599 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 23756
2026-08-09T02:16:11Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (359 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (125 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (126 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:16:12 dist-test-slave-s11t krb5kdc[23929](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:16:12 dist-test-slave-s11t krb5kdc[23929](info): set up 2 sockets
Aug 09 02:16:12 dist-test-slave-s11t krb5kdc[23929](info): commencing operation
krb5kdc: starting...
W20260809 02:16:13.469404 15662 mini_kdc.cc:121] Time spent starting KDC: real 1.208s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Aug 09 02:16:13 dist-test-slave-s11t krb5kdc[23929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241773, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:16:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:16:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.75.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab.
Entry for principal kudu/127.15.75.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab.
I20260809 02:16:13.593189 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:45887
--webserver_interface=127.15.75.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab
--principal=kudu/127.15.75.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:43431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.75.190:45887
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:16:13.705132 23950 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:16:13.705435 23950 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:16:13.705477 23950 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:16:13.709477 23950 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:16:13.709537 23950 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:16:13.709556 23950 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:16:13.709573 23950 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:16:13.709587 23950 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:16:13.715001 23950 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:43431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.75.190:45887
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab
--principal=kudu/127.15.75.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:45887
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.75.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23950
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:16:13.716087 23950 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:16:13.717127 23950 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:16:13.725713 23958 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:13.725759 23957 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:13.725921 23950 server_base.cc:1034] running on GCE node
W20260809 02:16:13.725759 23960 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:13.726593 23950 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:16:13.727676 23950 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:16:13.728833 23950 hybrid_clock.cc:648] HybridClock initialized: now 1786241773728804 us; error 37 us; skew 500 ppm
Aug 09 02:16:13 dist-test-slave-s11t krb5kdc[23929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241773, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:16:13.735914 23950 init.cc:377] Logged in from keytab as kudu/127.15.75.190@KRBTEST.COM (short username kudu)
I20260809 02:16:13.736900 23950 webserver.cc:460] Webserver started at http://127.15.75.190:46395/ using document root <none> and password file <none>
I20260809 02:16:13.737476 23950 fs_manager.cc:359] Metadata directory not provided
I20260809 02:16:13.737538 23950 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:16:13.737782 23950 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:16:13.739897 23950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/minicluster-data/master-0/data/instance:
uuid: "1844fddb6faf41359180e51529734084"
format_stamp: "Formatted at 2026-08-09 02:16:13 on dist-test-slave-s11t"
server_key: "9020bd5e086040b294e467fe78fd890b"
server_key_iv: "1753fe32920c7fe8c276b0e7275da214"
server_key_version: "encryptionkey@0"
I20260809 02:16:13.740473 23950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/minicluster-data/master-0/wal/instance:
uuid: "1844fddb6faf41359180e51529734084"
format_stamp: "Formatted at 2026-08-09 02:16:13 on dist-test-slave-s11t"
server_key: "9020bd5e086040b294e467fe78fd890b"
server_key_iv: "1753fe32920c7fe8c276b0e7275da214"
server_key_version: "encryptionkey@0"
I20260809 02:16:13.744266 23950 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260809 02:16:13.746765 23971 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:13.747975 23950 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 02:16:13.748111 23950 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/minicluster-data/master-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/minicluster-data/master-0/wal
uuid: "1844fddb6faf41359180e51529734084"
format_stamp: "Formatted at 2026-08-09 02:16:13 on dist-test-slave-s11t"
server_key: "9020bd5e086040b294e467fe78fd890b"
server_key_iv: "1753fe32920c7fe8c276b0e7275da214"
server_key_version: "encryptionkey@0"
I20260809 02:16:13.748221 23950 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:16:13.770258 23950 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:16:13.773381 23950 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:16:13.773595 23950 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:16:13.781019 23950 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.190:45887
I20260809 02:16:13.781047 24056 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.190:45887 every 8 connection(s)
I20260809 02:16:13.782063 23950 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
I20260809 02:16:13.785022 24057 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:16:13.789587 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 23950
I20260809 02:16:13.789732 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/minicluster-data/master-0/wal/instance
I20260809 02:16:13.790047 15662 external_mini_cluster.cc:1428] Setting key ba0a9774224a6a98bece4dd452d7a321
I20260809 02:16:13.791600 24057 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1844fddb6faf41359180e51529734084: Bootstrap starting.
I20260809 02:16:13.794121 24057 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1844fddb6faf41359180e51529734084: Neither blocks nor log segments found. Creating new log.
I20260809 02:16:13.794919 24057 log.cc:824] T 00000000000000000000000000000000 P 1844fddb6faf41359180e51529734084: Log is configured to *not* fsync() on all Append() calls
I20260809 02:16:13.796927 24057 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1844fddb6faf41359180e51529734084: No bootstrap required, opened a new log
Aug 09 02:16:13 dist-test-slave-s11t krb5kdc[23929](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241773, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:16:13.801213 24057 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1844fddb6faf41359180e51529734084 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1844fddb6faf41359180e51529734084" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 45887 } }
I20260809 02:16:13.801446 24057 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1844fddb6faf41359180e51529734084 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:16:13.801530 24057 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1844fddb6faf41359180e51529734084 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1844fddb6faf41359180e51529734084, State: Initialized, Role: FOLLOWER
I20260809 02:16:13.801980 24057 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1844fddb6faf41359180e51529734084 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1844fddb6faf41359180e51529734084" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 45887 } }
I20260809 02:16:13.802165 24057 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1844fddb6faf41359180e51529734084 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:16:13.802243 24057 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1844fddb6faf41359180e51529734084 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:16:13.802371 24057 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1844fddb6faf41359180e51529734084 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:16:13.803541 24057 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1844fddb6faf41359180e51529734084 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1844fddb6faf41359180e51529734084" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 45887 } }
I20260809 02:16:13.803936 24057 leader_election.cc:302] T 00000000000000000000000000000000 P 1844fddb6faf41359180e51529734084 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1844fddb6faf41359180e51529734084; no voters:
I20260809 02:16:13.804345 24057 leader_election.cc:288] T 00000000000000000000000000000000 P 1844fddb6faf41359180e51529734084 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:16:13.804404 24066 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1844fddb6faf41359180e51529734084 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:16:13.804638 24066 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1844fddb6faf41359180e51529734084 [term 1 LEADER]: Becoming Leader. State: Replica: 1844fddb6faf41359180e51529734084, State: Running, Role: LEADER
I20260809 02:16:13.805019 24066 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1844fddb6faf41359180e51529734084 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1844fddb6faf41359180e51529734084" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 45887 } }
I20260809 02:16:13.805143 24061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:13.791663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42346 (local address 127.15.75.190:45887)
0809 02:16:13.792137 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:13.792153 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:13.792188 (+ 35us) server_negotiation.cc:407] Connection header received
0809 02:16:13.792887 (+ 699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:13.792906 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:13.793234 (+ 328us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:13.793571 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:13.794460 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:13.795481 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:13.796324 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:13.796849 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:13.797034 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:13.799974 (+ 2940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:13.799979 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:13.799990 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:13.800016 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:13.801969 (+ 1953us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:13.802501 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:13.802507 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:13.802512 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:13.802636 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:13.802981 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:13.802984 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:13.802985 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:13.803366 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:13.803509 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:13.803894 (+ 385us) server_negotiation.cc:299] Negotiation successful
0809 02:16:13.804197 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":157,"threads_started":1}
I20260809 02:16:13.805709 24057 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1844fddb6faf41359180e51529734084 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:16:13.806277 24066 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1844fddb6faf41359180e51529734084 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1844fddb6faf41359180e51529734084. Latest consensus state: current_term: 1 leader_uuid: "1844fddb6faf41359180e51529734084" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1844fddb6faf41359180e51529734084" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 45887 } } }
I20260809 02:16:13.806463 24066 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1844fddb6faf41359180e51529734084 [sys.catalog]: This master's current role is: LEADER
I20260809 02:16:13.806676 24067 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1844fddb6faf41359180e51529734084 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1844fddb6faf41359180e51529734084" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1844fddb6faf41359180e51529734084" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 45887 } } }
I20260809 02:16:13.806787 24067 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1844fddb6faf41359180e51529734084 [sys.catalog]: This master's current role is: LEADER
I20260809 02:16:13.807408 24076 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:16:13.810845 24076 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:16:13.816221 24076 catalog_manager.cc:1347] Generated new cluster ID: ff820718b0664ef9bd8aed72c4994e47
I20260809 02:16:13.816288 24076 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:16:13.844475 24076 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:16:13.845701 24076 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:16:13.851747 24076 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1844fddb6faf41359180e51529734084: Generated new TSK 0
I20260809 02:16:13.852480 24076 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.75.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab.
Entry for principal kudu/127.15.75.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab.
I20260809 02:16:13.880899 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.129:0
--local_ip_for_outbound_sockets=127.15.75.129
--webserver_interface=127.15.75.129
--webserver_port=0
--tserver_master_addrs=127.15.75.190:45887
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab
--principal=kudu/127.15.75.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:43431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:16:13.990366 24098 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:16:13.990590 24098 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:16:13.990629 24098 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:16:13.994577 24098 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:16:13.994626 24098 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:16:13.994681 24098 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.129
I20260809 02:16:13.999683 24098 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:43431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab
--principal=kudu/127.15.75.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.75.129
--webserver_port=0
--tserver_master_addrs=127.15.75.190:45887
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24098
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.129
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:16:14.000634 24098 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:16:14.001614 24098 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:16:14.010977 24106 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:14.010987 24107 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:14.011024 24098 server_base.cc:1034] running on GCE node
W20260809 02:16:14.010993 24113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:14.011535 24098 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:16:14.012161 24098 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:16:14.013339 24098 hybrid_clock.cc:648] HybridClock initialized: now 1786241774013325 us; error 28 us; skew 500 ppm
Aug 09 02:16:14 dist-test-slave-s11t krb5kdc[23929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241774, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:16:14.016139 24098 init.cc:377] Logged in from keytab as kudu/127.15.75.129@KRBTEST.COM (short username kudu)
I20260809 02:16:14.017016 24098 webserver.cc:460] Webserver started at http://127.15.75.129:45245/ using document root <none> and password file <none>
I20260809 02:16:14.017508 24098 fs_manager.cc:359] Metadata directory not provided
I20260809 02:16:14.017566 24098 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:16:14.017721 24098 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:16:14.019730 24098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/minicluster-data/ts-0/data/instance:
uuid: "0d638be81e874c3383512b106d7241a2"
format_stamp: "Formatted at 2026-08-09 02:16:14 on dist-test-slave-s11t"
server_key: "d2202ab2cd6bd30cd9e2a003d52ee783"
server_key_iv: "49506412f65747538a5d63fa33add190"
server_key_version: "encryptionkey@0"
I20260809 02:16:14.020244 24098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/minicluster-data/ts-0/wal/instance:
uuid: "0d638be81e874c3383512b106d7241a2"
format_stamp: "Formatted at 2026-08-09 02:16:14 on dist-test-slave-s11t"
server_key: "d2202ab2cd6bd30cd9e2a003d52ee783"
server_key_iv: "49506412f65747538a5d63fa33add190"
server_key_version: "encryptionkey@0"
I20260809 02:16:14.023962 24098 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260809 02:16:14.026855 24124 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:14.028083 24098 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 02:16:14.028211 24098 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/minicluster-data/ts-0/wal
uuid: "0d638be81e874c3383512b106d7241a2"
format_stamp: "Formatted at 2026-08-09 02:16:14 on dist-test-slave-s11t"
server_key: "d2202ab2cd6bd30cd9e2a003d52ee783"
server_key_iv: "49506412f65747538a5d63fa33add190"
server_key_version: "encryptionkey@0"
I20260809 02:16:14.028323 24098 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:16:14.041488 24098 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:16:14.044413 24098 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:16:14.044631 24098 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:16:14.045289 24098 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:16:14.046110 24098 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:16:14.046164 24098 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:14.046209 24098 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:16:14.046227 24098 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:14.056052 24098 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.129:43073
I20260809 02:16:14.056077 24304 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.129:43073 every 8 connection(s)
I20260809 02:16:14.057077 24098 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
Aug 09 02:16:14 dist-test-slave-s11t krb5kdc[23929](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241774, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.129@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:16:14.066964 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 24098
I20260809 02:16:14.067098 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/minicluster-data/ts-0/wal/instance
I20260809 02:16:14.067406 15662 external_mini_cluster.cc:1428] Setting key f80a0098e741f926f3c88a29ff04cda9
I20260809 02:16:14.071506 24061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:14.058874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.129:53557 (local address 127.15.75.190:45887)
0809 02:16:14.059061 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:14.059065 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:14.059909 (+ 844us) server_negotiation.cc:407] Connection header received
0809 02:16:14.061333 (+ 1424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:14.061337 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:14.061398 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:14.061494 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:14.062618 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:14.063170 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:14.064319 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:14.064549 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:14.064659 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:14.067289 (+ 2630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:14.067292 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:14.067294 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:14.067320 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:14.069123 (+ 1803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:14.069730 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:14.069732 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:14.069734 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:14.069853 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:14.070254 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:14.070256 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:14.070258 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:14.070426 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:14.070499 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:14.071132 (+ 633us) server_negotiation.cc:299] Negotiation successful
0809 02:16:14.071280 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260809 02:16:14.072371 24309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:14.059280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:45887 (local address 127.15.75.129:53557)
0809 02:16:14.059757 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:14.059800 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:14.061120 (+ 1320us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:14.061660 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:14.061666 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:14.062019 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:14.062479 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:14.062491 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:14.063307 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:14.063310 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:14.064165 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:14.064173 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:14.064730 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:14.064735 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:14.064899 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:14.065482 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:14.065498 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:14.067079 (+ 1581us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:16:14.069243 (+ 2164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:14.069250 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:14.069264 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:14.069633 (+ 369us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:14.069953 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:14.069958 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:14.069963 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:14.070162 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:14.070526 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:16:14.070537 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:16:14.070864 (+ 327us) client_negotiation.cc:769] Sending connection context
0809 02:16:14.071099 (+ 235us) client_negotiation.cc:240] Negotiation successful
0809 02:16:14.071482 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":124,"threads_started":1}
I20260809 02:16:14.073565 24306 heartbeater.cc:347] Connected to a master server at 127.15.75.190:45887
I20260809 02:16:14.073835 24306 heartbeater.cc:464] Registering TS with master...
I20260809 02:16:14.074460 24306 heartbeater.cc:511] Master 127.15.75.190:45887 requested a full tablet report, sending...
I20260809 02:16:14.076057 23994 ts_manager.cc:194] Registered new tserver with Master: 0d638be81e874c3383512b106d7241a2 (127.15.75.129:43073)
I20260809 02:16:14.077256 23994 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.75.129@KRBTEST.COM'} at 127.15.75.129:53557
WARNING: no policy specified for kudu/127.15.75.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab.
Entry for principal kudu/127.15.75.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab.
I20260809 02:16:14.095744 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.130:0
--local_ip_for_outbound_sockets=127.15.75.130
--webserver_interface=127.15.75.130
--webserver_port=0
--tserver_master_addrs=127.15.75.190:45887
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab
--principal=kudu/127.15.75.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:43431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:16:14.206156 24312 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:16:14.206400 24312 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:16:14.206442 24312 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:16:14.210541 24312 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:16:14.210590 24312 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:16:14.210644 24312 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.130
I20260809 02:16:14.215529 24312 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:43431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab
--principal=kudu/127.15.75.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.75.130
--webserver_port=0
--tserver_master_addrs=127.15.75.190:45887
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24312
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.130
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:16:14.216393 24312 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:16:14.217355 24312 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:16:14.226298 24322 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:14.226328 24324 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:14.226496 24319 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:14.226501 24312 server_base.cc:1034] running on GCE node
I20260809 02:16:14.226953 24312 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:16:14.227527 24312 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:16:14.228735 24312 hybrid_clock.cc:648] HybridClock initialized: now 1786241774228709 us; error 42 us; skew 500 ppm
Aug 09 02:16:14 dist-test-slave-s11t krb5kdc[23929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241774, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:16:14.231397 24312 init.cc:377] Logged in from keytab as kudu/127.15.75.130@KRBTEST.COM (short username kudu)
I20260809 02:16:14.232275 24312 webserver.cc:460] Webserver started at http://127.15.75.130:41311/ using document root <none> and password file <none>
I20260809 02:16:14.232869 24312 fs_manager.cc:359] Metadata directory not provided
I20260809 02:16:14.232957 24312 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:16:14.233222 24312 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:16:14.235158 24312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/minicluster-data/ts-1/data/instance:
uuid: "2be4ccb209354c29a8f43e8fde8230e0"
format_stamp: "Formatted at 2026-08-09 02:16:14 on dist-test-slave-s11t"
server_key: "d15a4561c3bac3bce71b0f5ef5979bfc"
server_key_iv: "35317d3f6c41bd3016250d0ac55aa02b"
server_key_version: "encryptionkey@0"
I20260809 02:16:14.235709 24312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/minicluster-data/ts-1/wal/instance:
uuid: "2be4ccb209354c29a8f43e8fde8230e0"
format_stamp: "Formatted at 2026-08-09 02:16:14 on dist-test-slave-s11t"
server_key: "d15a4561c3bac3bce71b0f5ef5979bfc"
server_key_iv: "35317d3f6c41bd3016250d0ac55aa02b"
server_key_version: "encryptionkey@0"
I20260809 02:16:14.239223 24312 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.006s
I20260809 02:16:14.241546 24333 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:14.242662 24312 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 02:16:14.242823 24312 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/minicluster-data/ts-1/wal
uuid: "2be4ccb209354c29a8f43e8fde8230e0"
format_stamp: "Formatted at 2026-08-09 02:16:14 on dist-test-slave-s11t"
server_key: "d15a4561c3bac3bce71b0f5ef5979bfc"
server_key_iv: "35317d3f6c41bd3016250d0ac55aa02b"
server_key_version: "encryptionkey@0"
I20260809 02:16:14.242929 24312 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:16:14.273823 24312 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:16:14.277024 24312 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:16:14.277237 24312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:16:14.277855 24312 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:16:14.278709 24312 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:16:14.278795 24312 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:14.278865 24312 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:16:14.278913 24312 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:14.287974 24312 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.130:41419
I20260809 02:16:14.288004 24504 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.130:41419 every 8 connection(s)
I20260809 02:16:14.288879 24312 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
I20260809 02:16:14.291985 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 24312
I20260809 02:16:14.292104 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/minicluster-data/ts-1/wal/instance
I20260809 02:16:14.292434 15662 external_mini_cluster.cc:1428] Setting key fb706f4be990e996cd312574dfbdb1d6
Aug 09 02:16:14 dist-test-slave-s11t krb5kdc[23929](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241774, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.130@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.75.131@KRBTEST.COM; defaulting to no policy
I20260809 02:16:14.303979 24061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:14.290685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.130:42115 (local address 127.15.75.190:45887)
0809 02:16:14.290845 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:14.290850 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:14.291731 (+ 881us) server_negotiation.cc:407] Connection header received
0809 02:16:14.293487 (+ 1756us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:14.293490 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:14.293535 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:14.293620 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:14.295166 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:14.295687 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:14.297121 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:14.297345 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:14.297454 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:14.300155 (+ 2701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:14.300161 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:14.300168 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:14.300207 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:14.301894 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:14.302388 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:14.302395 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:14.302398 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:14.302486 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:14.302827 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:14.302833 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:14.302837 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:14.303109 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:14.303223 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:14.303688 (+ 465us) server_negotiation.cc:299] Negotiation successful
0809 02:16:14.303844 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 02:16:14.304943 24507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:14.291115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:45887 (local address 127.15.75.130:42115)
0809 02:16:14.291578 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:14.291621 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:14.293279 (+ 1658us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:14.293804 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:14.293813 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:14.294311 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:14.295007 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:14.295030 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:14.295815 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:14.295822 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:14.296999 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:14.297009 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:14.297491 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:14.297496 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:14.297665 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:14.298256 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:14.298273 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:14.299977 (+ 1704us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:16:14.302013 (+ 2036us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:14.302018 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:14.302030 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:14.302281 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:14.302581 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:14.302584 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:14.302585 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:14.302722 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:14.303216 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:16:14.303221 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:16:14.303443 (+ 222us) client_negotiation.cc:769] Sending connection context
0809 02:16:14.303633 (+ 190us) client_negotiation.cc:240] Negotiation successful
0809 02:16:14.303910 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":109,"threads_started":1}
I20260809 02:16:14.306118 24505 heartbeater.cc:347] Connected to a master server at 127.15.75.190:45887
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.131@KRBTEST.COM" created.
I20260809 02:16:14.306420 24505 heartbeater.cc:464] Registering TS with master...
I20260809 02:16:14.307037 24505 heartbeater.cc:511] Master 127.15.75.190:45887 requested a full tablet report, sending...
I20260809 02:16:14.308251 23994 ts_manager.cc:194] Registered new tserver with Master: 2be4ccb209354c29a8f43e8fde8230e0 (127.15.75.130:41419)
I20260809 02:16:14.308910 23994 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.75.130@KRBTEST.COM'} at 127.15.75.130:42115
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab.
Entry for principal kudu/127.15.75.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab.
I20260809 02:16:14.321874 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.131:0
--local_ip_for_outbound_sockets=127.15.75.131
--webserver_interface=127.15.75.131
--webserver_port=0
--tserver_master_addrs=127.15.75.190:45887
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab
--principal=kudu/127.15.75.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:43431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:16:14.435530 24512 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:16:14.435770 24512 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:16:14.435846 24512 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:16:14.439886 24512 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:16:14.439937 24512 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:16:14.439993 24512 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.131
I20260809 02:16:14.444972 24512 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:43431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab
--principal=kudu/127.15.75.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.75.131
--webserver_port=0
--tserver_master_addrs=127.15.75.190:45887
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24512
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.131
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:16:14.445946 24512 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:16:14.446980 24512 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:16:14.456573 24520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:14.456578 24521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:14.456677 24512 server_base.cc:1034] running on GCE node
W20260809 02:16:14.456583 24523 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:14.457202 24512 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:16:14.457834 24512 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:16:14.458997 24512 hybrid_clock.cc:648] HybridClock initialized: now 1786241774458981 us; error 27 us; skew 500 ppm
Aug 09 02:16:14 dist-test-slave-s11t krb5kdc[23929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241774, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:16:14.461807 24512 init.cc:377] Logged in from keytab as kudu/127.15.75.131@KRBTEST.COM (short username kudu)
I20260809 02:16:14.462601 24512 webserver.cc:460] Webserver started at http://127.15.75.131:40203/ using document root <none> and password file <none>
I20260809 02:16:14.463140 24512 fs_manager.cc:359] Metadata directory not provided
I20260809 02:16:14.463200 24512 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:16:14.463446 24512 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:16:14.465464 24512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/minicluster-data/ts-2/data/instance:
uuid: "b692857d5ced4b74a6305719a0aabfbf"
format_stamp: "Formatted at 2026-08-09 02:16:14 on dist-test-slave-s11t"
server_key: "7a7ead302dce6c6e2904fec9ed9d57a1"
server_key_iv: "710f9fd704c795488a76b1ab7e0c92b5"
server_key_version: "encryptionkey@0"
I20260809 02:16:14.465997 24512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/minicluster-data/ts-2/wal/instance:
uuid: "b692857d5ced4b74a6305719a0aabfbf"
format_stamp: "Formatted at 2026-08-09 02:16:14 on dist-test-slave-s11t"
server_key: "7a7ead302dce6c6e2904fec9ed9d57a1"
server_key_iv: "710f9fd704c795488a76b1ab7e0c92b5"
server_key_version: "encryptionkey@0"
I20260809 02:16:14.469519 24512 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 02:16:14.471767 24531 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:14.472944 24512 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 02:16:14.473063 24512 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/minicluster-data/ts-2/wal
uuid: "b692857d5ced4b74a6305719a0aabfbf"
format_stamp: "Formatted at 2026-08-09 02:16:14 on dist-test-slave-s11t"
server_key: "7a7ead302dce6c6e2904fec9ed9d57a1"
server_key_iv: "710f9fd704c795488a76b1ab7e0c92b5"
server_key_version: "encryptionkey@0"
I20260809 02:16:14.473162 24512 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:16:14.486117 24512 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:16:14.489310 24512 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:16:14.489516 24512 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:16:14.490062 24512 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:16:14.490844 24512 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:16:14.490912 24512 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:14.490978 24512 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:16:14.491024 24512 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:14.500097 24512 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.131:41023
I20260809 02:16:14.500118 24711 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.131:41023 every 8 connection(s)
I20260809 02:16:14.500975 24512 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
I20260809 02:16:14.508183 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 24512
I20260809 02:16:14.508306 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/minicluster-data/ts-2/wal/instance
I20260809 02:16:14.508658 15662 external_mini_cluster.cc:1428] Setting key 5054871a07e44644032ed4e3c7b77d8b
Aug 09 02:16:14 dist-test-slave-s11t krb5kdc[23929](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241774, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.131@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:16:14.514715 24061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:14.502796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.131:48033 (local address 127.15.75.190:45887)
0809 02:16:14.502970 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:14.502975 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:14.503788 (+ 813us) server_negotiation.cc:407] Connection header received
0809 02:16:14.505167 (+ 1379us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:14.505170 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:14.505217 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:14.505281 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:14.506484 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:14.507140 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:14.508668 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:14.508961 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:14.509071 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:14.511621 (+ 2550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:14.511624 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:14.511626 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:14.511650 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:14.512927 (+ 1277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:14.513413 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:14.513415 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:14.513417 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:14.513491 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:14.513828 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:14.513830 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:14.513831 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:14.514003 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:14.514079 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:14.514504 (+ 425us) server_negotiation.cc:299] Negotiation successful
0809 02:16:14.514598 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 02:16:14.515466 24716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:14.503152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:45887 (local address 127.15.75.131:48033)
0809 02:16:14.503609 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:14.503651 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:14.504957 (+ 1306us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:14.505442 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:14.505448 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:14.505883 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:14.506352 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:14.506365 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:14.507276 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:14.507279 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:14.508550 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:14.508562 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:14.509115 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:14.509120 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:14.509281 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:14.509915 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:14.509932 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:14.511479 (+ 1547us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:16:14.513036 (+ 1557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:14.513041 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:14.513052 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:14.513292 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:14.513568 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:14.513571 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:14.513572 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:14.513748 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:14.514090 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:16:14.514094 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:16:14.514303 (+ 209us) client_negotiation.cc:769] Sending connection context
0809 02:16:14.514457 (+ 154us) client_negotiation.cc:240] Negotiation successful
0809 02:16:14.514687 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":112,"threads_started":1}
I20260809 02:16:14.516523 24712 heartbeater.cc:347] Connected to a master server at 127.15.75.190:45887
I20260809 02:16:14.516785 24712 heartbeater.cc:464] Registering TS with master...
I20260809 02:16:14.517253 24712 heartbeater.cc:511] Master 127.15.75.190:45887 requested a full tablet report, sending...
I20260809 02:16:14.518245 23994 ts_manager.cc:194] Registered new tserver with Master: b692857d5ced4b74a6305719a0aabfbf (127.15.75.131:41023)
I20260809 02:16:14.518755 23994 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.75.131@KRBTEST.COM'} at 127.15.75.131:48033
I20260809 02:16:14.522933 15662 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/krb5kdc/test-admin.keytab.
Aug 09 02:16:14 dist-test-slave-s11t krb5kdc[23929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241774, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:16:14.538921 15662 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260809 02:16:14.541672 24722 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:16:14.540222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43222 (local address 127.15.75.129:43073)
0809 02:16:14.540555 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:14.540570 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:14.540593 (+ 23us) server_negotiation.cc:407] Connection header received
0809 02:16:14.540644 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:14.540648 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:14.540892 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:14.541014 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:14.541473 (+ 459us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43222: BlockingRecv error: recv got EOF from 127.0.0.1:43222 (error 108)
Metrics: {"server-negotiator.queue_time_us":234,"thread_start_us":151,"threads_started":1}
W20260809 02:16:14.541752 15662 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/krb5kdc/krb5cc' of type FILE
Aug 09 02:16:14 dist-test-slave-s11t krb5kdc[23929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241774, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:16:14.542762 15662 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 02:16:14 dist-test-slave-s11t krb5kdc[23929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241774, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:16:14.543738 15662 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 02:16:14 dist-test-slave-s11t krb5kdc[23929](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241774, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.75.129@KRBTEST.COM
I20260809 02:16:14.553316 24722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:14.544576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43236 (local address 127.15.75.129:43073)
0809 02:16:14.544739 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:14.544743 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:14.544844 (+ 101us) server_negotiation.cc:407] Connection header received
0809 02:16:14.544935 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:14.544937 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:14.544975 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:14.545041 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:14.545539 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:14.546393 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:14.547257 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:14.547618 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:14.547699 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:14.549876 (+ 2177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:14.549885 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:14.549899 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:14.549920 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:14.551509 (+ 1589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:14.551940 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:14.551945 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:14.551947 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:14.552032 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:14.552312 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:14.552316 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:14.552318 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:14.552562 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:14.552829 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:14.553024 (+ 195us) server_negotiation.cc:299] Negotiation successful
0809 02:16:14.553189 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
W20260809 02:16:14.556773 24722 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:16:14.555913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43240 (local address 127.15.75.129:43073)
0809 02:16:14.556016 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:14.556020 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:14.556148 (+ 128us) server_negotiation.cc:407] Connection header received
0809 02:16:14.556233 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:14.556235 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:14.556267 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:14.556329 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:14.556649 (+ 320us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43240: BlockingRecv error: recv got EOF from 127.0.0.1:43240 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260809 02:16:14.556927 15662 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/krb5kdc/krb5cc' of type FILE
Aug 09 02:16:14 dist-test-slave-s11t krb5kdc[23929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241774, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:16:14.557830 15662 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 02:16:14 dist-test-slave-s11t krb5kdc[23929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241774, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:16:14.558714 15662 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 02:16:14 dist-test-slave-s11t krb5kdc[23929](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241774, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.75.129@KRBTEST.COM
I20260809 02:16:14.567004 24722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:14.559576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43250 (local address 127.15.75.129:43073)
0809 02:16:14.559695 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:14.559699 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:14.559808 (+ 109us) server_negotiation.cc:407] Connection header received
0809 02:16:14.559946 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:14.559949 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:14.559989 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:14.560061 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:14.560520 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:14.561106 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:14.561866 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:14.562101 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:14.562200 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:14.564227 (+ 2027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:14.564231 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:14.564234 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:14.564256 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:14.565498 (+ 1242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:14.565914 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:14.565921 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:14.565923 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:14.565991 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:14.566253 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:14.566256 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:14.566258 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:14.566410 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:14.566536 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:14.566761 (+ 225us) server_negotiation.cc:299] Negotiation successful
0809 02:16:14.566875 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
W20260809 02:16:14.569379 24722 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:16:14.568505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43262 (local address 127.15.75.129:43073)
0809 02:16:14.568631 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:14.568634 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:14.568661 (+ 27us) server_negotiation.cc:407] Connection header received
0809 02:16:14.568833 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:14.568835 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:14.568870 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:14.568931 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:14.569283 (+ 352us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43262: BlockingRecv error: recv got EOF from 127.0.0.1:43262 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260809 02:16:14.569446 15662 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241741788589-15662-0/krb5kdc/krb5cc' of type FILE
Aug 09 02:16:14 dist-test-slave-s11t krb5kdc[23929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241774, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:16:14.570511 15662 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 02:16:14 dist-test-slave-s11t krb5kdc[23929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241774, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:16:14.571341 15662 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 02:16:14 dist-test-slave-s11t krb5kdc[23929](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241774, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.75.129@KRBTEST.COM
I20260809 02:16:14.579332 24722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:14.572070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43272 (local address 127.15.75.129:43073)
0809 02:16:14.572203 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:14.572206 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:14.572218 (+ 12us) server_negotiation.cc:407] Connection header received
0809 02:16:14.572314 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:14.572316 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:14.572346 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:14.572408 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:14.572926 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:14.573384 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:14.574115 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:14.574337 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:14.574416 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:14.576457 (+ 2041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:14.576460 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:14.576462 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:14.576483 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:14.577996 (+ 1513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:14.578356 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:14.578359 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:14.578360 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:14.578440 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:14.578650 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:14.578652 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:14.578653 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:14.578791 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:14.578883 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:14.579062 (+ 179us) server_negotiation.cc:299] Negotiation successful
0809 02:16:14.579173 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 02:16:14.580323 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 24098
I20260809 02:16:14.592505 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 24312
I20260809 02:16:14.606799 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 24512
I20260809 02:16:14.616503 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 23950
2026-08-09T02:16:14Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2379 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:16:14 dist-test-slave-s11t krb5kdc[24745](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:16:14 dist-test-slave-s11t krb5kdc[24745](info): set up 2 sockets
Aug 09 02:16:14 dist-test-slave-s11t krb5kdc[24745](info): commencing operation
krb5kdc: starting...
W20260809 02:16:15.852000 15662 mini_kdc.cc:121] Time spent starting KDC: real 1.211s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Aug 09 02:16:15 dist-test-slave-s11t krb5kdc[24745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241775, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:16:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:16:15Z Disabled control of system clock
WARNING: no policy specified for oryx/127.15.75.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.75.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.75.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/krb5kdc/oryx_127.15.75.190.keytab.
Entry for principal oryx/127.15.75.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/krb5kdc/oryx_127.15.75.190.keytab.
I20260809 02:16:15.971598 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:42501
--webserver_interface=127.15.75.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/krb5kdc/oryx_127.15.75.190.keytab
--principal=oryx/127.15.75.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:34317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.75.190:42501
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:16:16.084026 24763 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:16:16.084260 24763 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:16:16.084297 24763 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:16:16.088126 24763 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:16:16.088176 24763 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:16:16.088194 24763 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:16:16.088210 24763 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:16:16.088225 24763 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:16:16.093204 24763 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:34317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.75.190:42501
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/krb5kdc/oryx_127.15.75.190.keytab
--principal=oryx/127.15.75.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:42501
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.75.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24763
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:16:16.094087 24763 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:16:16.094985 24763 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:16:16.103104 24770 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:16.103117 24768 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:16.103209 24763 server_base.cc:1034] running on GCE node
W20260809 02:16:16.103147 24772 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:16.103857 24763 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:16:16.104972 24763 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:16:16.106127 24763 hybrid_clock.cc:648] HybridClock initialized: now 1786241776106100 us; error 41 us; skew 500 ppm
Aug 09 02:16:16 dist-test-slave-s11t krb5kdc[24745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241776, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.75.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:16:16.108839 24763 init.cc:377] Logged in from keytab as oryx/127.15.75.190@KRBTEST.COM (short username oryx)
I20260809 02:16:16.109705 24763 webserver.cc:460] Webserver started at http://127.15.75.190:39561/ using document root <none> and password file <none>
I20260809 02:16:16.110255 24763 fs_manager.cc:359] Metadata directory not provided
I20260809 02:16:16.110316 24763 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:16:16.110543 24763 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:16:16.112528 24763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/master-0/data/instance:
uuid: "c59a0fdc35554648bd22e9b2280fe648"
format_stamp: "Formatted at 2026-08-09 02:16:16 on dist-test-slave-s11t"
server_key: "14aacedaa86a070c565722740ac0c4ba"
server_key_iv: "63fe2718bcdf256bd2242a1464ad3f25"
server_key_version: "encryptionkey@0"
I20260809 02:16:16.113148 24763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/master-0/wal/instance:
uuid: "c59a0fdc35554648bd22e9b2280fe648"
format_stamp: "Formatted at 2026-08-09 02:16:16 on dist-test-slave-s11t"
server_key: "14aacedaa86a070c565722740ac0c4ba"
server_key_iv: "63fe2718bcdf256bd2242a1464ad3f25"
server_key_version: "encryptionkey@0"
I20260809 02:16:16.116726 24763 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 02:16:16.118981 24786 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:16.120147 24763 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 02:16:16.120270 24763 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/master-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/master-0/wal
uuid: "c59a0fdc35554648bd22e9b2280fe648"
format_stamp: "Formatted at 2026-08-09 02:16:16 on dist-test-slave-s11t"
server_key: "14aacedaa86a070c565722740ac0c4ba"
server_key_iv: "63fe2718bcdf256bd2242a1464ad3f25"
server_key_version: "encryptionkey@0"
I20260809 02:16:16.120365 24763 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:16:16.139688 24763 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:16:16.142508 24763 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:16:16.142665 24763 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:16:16.149633 24763 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.190:42501
I20260809 02:16:16.149919 24879 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.190:42501 every 8 connection(s)
I20260809 02:16:16.150612 24763 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
I20260809 02:16:16.153306 24880 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:16:16.157606 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 24763
I20260809 02:16:16.157737 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/master-0/wal/instance
I20260809 02:16:16.158056 15662 external_mini_cluster.cc:1428] Setting key 3e80e4f082402d267c7d085e20eaee90
I20260809 02:16:16.159368 24880 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c59a0fdc35554648bd22e9b2280fe648: Bootstrap starting.
I20260809 02:16:16.161852 24880 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c59a0fdc35554648bd22e9b2280fe648: Neither blocks nor log segments found. Creating new log.
I20260809 02:16:16.162632 24880 log.cc:824] T 00000000000000000000000000000000 P c59a0fdc35554648bd22e9b2280fe648: Log is configured to *not* fsync() on all Append() calls
I20260809 02:16:16.164438 24880 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c59a0fdc35554648bd22e9b2280fe648: No bootstrap required, opened a new log
Aug 09 02:16:16 dist-test-slave-s11t krb5kdc[24745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241775, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM
I20260809 02:16:16.168448 24880 raft_consensus.cc:348] T 00000000000000000000000000000000 P c59a0fdc35554648bd22e9b2280fe648 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c59a0fdc35554648bd22e9b2280fe648" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 42501 } }
I20260809 02:16:16.168764 24880 raft_consensus.cc:374] T 00000000000000000000000000000000 P c59a0fdc35554648bd22e9b2280fe648 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:16:16.168851 24880 raft_consensus.cc:729] T 00000000000000000000000000000000 P c59a0fdc35554648bd22e9b2280fe648 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c59a0fdc35554648bd22e9b2280fe648, State: Initialized, Role: FOLLOWER
I20260809 02:16:16.169312 24880 consensus_queue.cc:260] T 00000000000000000000000000000000 P c59a0fdc35554648bd22e9b2280fe648 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c59a0fdc35554648bd22e9b2280fe648" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 42501 } }
I20260809 02:16:16.169492 24880 raft_consensus.cc:388] T 00000000000000000000000000000000 P c59a0fdc35554648bd22e9b2280fe648 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:16:16.169564 24880 raft_consensus.cc:482] T 00000000000000000000000000000000 P c59a0fdc35554648bd22e9b2280fe648 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:16:16.169663 24880 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c59a0fdc35554648bd22e9b2280fe648 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:16:16.170734 24880 raft_consensus.cc:504] T 00000000000000000000000000000000 P c59a0fdc35554648bd22e9b2280fe648 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c59a0fdc35554648bd22e9b2280fe648" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 42501 } }
I20260809 02:16:16.171100 24880 leader_election.cc:302] T 00000000000000000000000000000000 P c59a0fdc35554648bd22e9b2280fe648 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c59a0fdc35554648bd22e9b2280fe648; no voters:
I20260809 02:16:16.171456 24880 leader_election.cc:288] T 00000000000000000000000000000000 P c59a0fdc35554648bd22e9b2280fe648 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:16:16.171550 24886 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c59a0fdc35554648bd22e9b2280fe648 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:16:16.171819 24886 raft_consensus.cc:686] T 00000000000000000000000000000000 P c59a0fdc35554648bd22e9b2280fe648 [term 1 LEADER]: Becoming Leader. State: Replica: c59a0fdc35554648bd22e9b2280fe648, State: Running, Role: LEADER
I20260809 02:16:16.172169 24886 consensus_queue.cc:237] T 00000000000000000000000000000000 P c59a0fdc35554648bd22e9b2280fe648 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c59a0fdc35554648bd22e9b2280fe648" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 42501 } }
I20260809 02:16:16.172715 24880 sys_catalog.cc:564] T 00000000000000000000000000000000 P c59a0fdc35554648bd22e9b2280fe648 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:16:16.173026 24883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:16.159454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59516 (local address 127.15.75.190:42501)
0809 02:16:16.159883 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:16.159899 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:16.159935 (+ 36us) server_negotiation.cc:407] Connection header received
0809 02:16:16.160563 (+ 628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:16.160580 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:16.160953 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:16.161222 (+ 269us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:16.162161 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:16.163228 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:16.164073 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:16.164561 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:16.164762 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:16.167609 (+ 2847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:16.167614 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:16.167627 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:16.167653 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:16.169615 (+ 1962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:16.170123 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:16.170127 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:16.170131 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:16.170251 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:16.170545 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:16.170548 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:16.170549 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:16.170981 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:16.171132 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:16.171495 (+ 363us) server_negotiation.cc:299] Negotiation successful
0809 02:16:16.172044 (+ 549us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":132,"threads_started":1}
I20260809 02:16:16.174094 24887 sys_catalog.cc:455] T 00000000000000000000000000000000 P c59a0fdc35554648bd22e9b2280fe648 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c59a0fdc35554648bd22e9b2280fe648" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c59a0fdc35554648bd22e9b2280fe648" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 42501 } } }
I20260809 02:16:16.174223 24887 sys_catalog.cc:458] T 00000000000000000000000000000000 P c59a0fdc35554648bd22e9b2280fe648 [sys.catalog]: This master's current role is: LEADER
I20260809 02:16:16.174518 24888 sys_catalog.cc:455] T 00000000000000000000000000000000 P c59a0fdc35554648bd22e9b2280fe648 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c59a0fdc35554648bd22e9b2280fe648. Latest consensus state: current_term: 1 leader_uuid: "c59a0fdc35554648bd22e9b2280fe648" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c59a0fdc35554648bd22e9b2280fe648" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 42501 } } }
I20260809 02:16:16.174659 24888 sys_catalog.cc:458] T 00000000000000000000000000000000 P c59a0fdc35554648bd22e9b2280fe648 [sys.catalog]: This master's current role is: LEADER
I20260809 02:16:16.175618 24902 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:16:16.178186 24902 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:16:16.184374 24902 catalog_manager.cc:1347] Generated new cluster ID: 60284658a9354acb9a8cbfb97c102def
I20260809 02:16:16.184444 24902 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:16:16.194516 24902 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:16:16.195364 24902 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:16:16.202507 24902 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c59a0fdc35554648bd22e9b2280fe648: Generated new TSK 0
I20260809 02:16:16.203135 24902 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.15.75.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.75.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.75.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/krb5kdc/oryx_127.15.75.129.keytab.
Entry for principal oryx/127.15.75.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/krb5kdc/oryx_127.15.75.129.keytab.
I20260809 02:16:16.235713 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.129:0
--local_ip_for_outbound_sockets=127.15.75.129
--webserver_interface=127.15.75.129
--webserver_port=0
--tserver_master_addrs=127.15.75.190:42501
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/krb5kdc/oryx_127.15.75.129.keytab
--principal=oryx/127.15.75.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:34317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:16:16.385866 24916 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:16:16.386113 24916 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:16:16.386197 24916 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:16:16.390331 24916 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:16:16.390378 24916 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:16:16.390481 24916 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.129
I20260809 02:16:16.395489 24916 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:34317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/krb5kdc/oryx_127.15.75.129.keytab
--principal=oryx/127.15.75.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.75.129
--webserver_port=0
--tserver_master_addrs=127.15.75.190:42501
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24916
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.129
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:16:16.396442 24916 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:16:16.397452 24916 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:16:16.406734 24927 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:16.406772 24925 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:16.406797 24924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:16.407084 24916 server_base.cc:1034] running on GCE node
I20260809 02:16:16.407488 24916 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:16:16.408063 24916 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:16:16.409238 24916 hybrid_clock.cc:648] HybridClock initialized: now 1786241776409209 us; error 40 us; skew 500 ppm
Aug 09 02:16:16 dist-test-slave-s11t krb5kdc[24745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241776, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.75.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:16:16.411978 24916 init.cc:377] Logged in from keytab as oryx/127.15.75.129@KRBTEST.COM (short username oryx)
I20260809 02:16:16.412915 24916 webserver.cc:460] Webserver started at http://127.15.75.129:35407/ using document root <none> and password file <none>
I20260809 02:16:16.413452 24916 fs_manager.cc:359] Metadata directory not provided
I20260809 02:16:16.413539 24916 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:16:16.413759 24916 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:16:16.415680 24916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-0/data/instance:
uuid: "6babbe94e6d54368a5c7edf6c5847ba1"
format_stamp: "Formatted at 2026-08-09 02:16:16 on dist-test-slave-s11t"
server_key: "556b0a5c7d4e5e06bd12aa213a0f1405"
server_key_iv: "85a32203d1f6bf9343c64eec7351f41f"
server_key_version: "encryptionkey@0"
I20260809 02:16:16.416252 24916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-0/wal/instance:
uuid: "6babbe94e6d54368a5c7edf6c5847ba1"
format_stamp: "Formatted at 2026-08-09 02:16:16 on dist-test-slave-s11t"
server_key: "556b0a5c7d4e5e06bd12aa213a0f1405"
server_key_iv: "85a32203d1f6bf9343c64eec7351f41f"
server_key_version: "encryptionkey@0"
I20260809 02:16:16.419866 24916 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260809 02:16:16.422188 24939 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:16.423368 24916 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 02:16:16.423496 24916 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-0/wal
uuid: "6babbe94e6d54368a5c7edf6c5847ba1"
format_stamp: "Formatted at 2026-08-09 02:16:16 on dist-test-slave-s11t"
server_key: "556b0a5c7d4e5e06bd12aa213a0f1405"
server_key_iv: "85a32203d1f6bf9343c64eec7351f41f"
server_key_version: "encryptionkey@0"
I20260809 02:16:16.423599 24916 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:16:16.440848 24916 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:16:16.443516 24916 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:16:16.443722 24916 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:16:16.444276 24916 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:16:16.445103 24916 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:16:16.445170 24916 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:16.445241 24916 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:16:16.445281 24916 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:16.453887 24916 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.129:34105
I20260809 02:16:16.453935 25111 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.129:34105 every 8 connection(s)
I20260809 02:16:16.454722 24916 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
I20260809 02:16:16.463115 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 24916
I20260809 02:16:16.463275 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-0/wal/instance
I20260809 02:16:16.463587 15662 external_mini_cluster.cc:1428] Setting key 7f4120765764742c9738800b10253e2f
Aug 09 02:16:16 dist-test-slave-s11t krb5kdc[24745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241776, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.75.129@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM
I20260809 02:16:16.468634 24883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:16.456407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.129:41459 (local address 127.15.75.190:42501)
0809 02:16:16.456536 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:16.456539 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:16.457339 (+ 800us) server_negotiation.cc:407] Connection header received
0809 02:16:16.458825 (+ 1486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:16.458828 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:16.458879 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:16.458955 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:16.460063 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:16.460593 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:16.461711 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:16.461940 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:16.462019 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:16.464510 (+ 2491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:16.464514 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:16.464517 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:16.464542 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:16.466246 (+ 1704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:16.466882 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:16.466886 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:16.466887 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:16.466979 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:16.467411 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:16.467416 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:16.467419 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:16.467689 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:16.467808 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:16.468268 (+ 460us) server_negotiation.cc:299] Negotiation successful
0809 02:16:16.468412 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 02:16:16.469324 25117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:16.456738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:42501 (local address 127.15.75.129:41459)
0809 02:16:16.457183 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:16.457223 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:16.458603 (+ 1380us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:16.459148 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:16.459155 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:16.459487 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:16.459936 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:16.459949 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:16.460752 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:16.460755 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:16.461612 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:16.461621 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:16.462065 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:16.462068 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:16.462200 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:16.462749 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:16.462766 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:16.464351 (+ 1585us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:16:16.466367 (+ 2016us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:16.466376 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:16.466393 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:16.466763 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:16.467076 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:16.467082 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:16.467087 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:16.467274 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:16.467799 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:16:16.467804 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:16:16.468031 (+ 227us) client_negotiation.cc:769] Sending connection context
0809 02:16:16.468183 (+ 152us) client_negotiation.cc:240] Negotiation successful
0809 02:16:16.468429 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":104,"threads_started":1}
I20260809 02:16:16.470564 25113 heartbeater.cc:347] Connected to a master server at 127.15.75.190:42501
I20260809 02:16:16.470831 25113 heartbeater.cc:464] Registering TS with master...
I20260809 02:16:16.471438 25113 heartbeater.cc:511] Master 127.15.75.190:42501 requested a full tablet report, sending...
I20260809 02:16:16.473577 24810 ts_manager.cc:194] Registered new tserver with Master: 6babbe94e6d54368a5c7edf6c5847ba1 (127.15.75.129:34105)
WARNING: no policy specified for oryx/127.15.75.130@KRBTEST.COM; defaulting to no policy
I20260809 02:16:16.475284 24810 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.75.129@KRBTEST.COM'} at 127.15.75.129:41459
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.75.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.75.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/krb5kdc/oryx_127.15.75.130.keytab.
Entry for principal oryx/127.15.75.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/krb5kdc/oryx_127.15.75.130.keytab.
I20260809 02:16:16.493340 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.130:0
--local_ip_for_outbound_sockets=127.15.75.130
--webserver_interface=127.15.75.130
--webserver_port=0
--tserver_master_addrs=127.15.75.190:42501
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/krb5kdc/oryx_127.15.75.130.keytab
--principal=oryx/127.15.75.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:34317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:16:16.602778 25126 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:16:16.603097 25126 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:16:16.603169 25126 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:16:16.607138 25126 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:16:16.607224 25126 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:16:16.607323 25126 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.130
I20260809 02:16:16.612416 25126 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:34317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/krb5kdc/oryx_127.15.75.130.keytab
--principal=oryx/127.15.75.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.75.130
--webserver_port=0
--tserver_master_addrs=127.15.75.190:42501
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25126
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.130
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:16:16.613564 25126 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:16:16.614595 25126 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:16:16.624121 25132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:16.624094 25133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:16.624329 25126 server_base.cc:1034] running on GCE node
W20260809 02:16:16.624183 25136 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:16.624930 25126 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:16:16.625566 25126 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:16:16.626741 25126 hybrid_clock.cc:648] HybridClock initialized: now 1786241776626727 us; error 33 us; skew 500 ppm
Aug 09 02:16:16 dist-test-slave-s11t krb5kdc[24745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241776, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.75.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:16:16.629566 25126 init.cc:377] Logged in from keytab as oryx/127.15.75.130@KRBTEST.COM (short username oryx)
I20260809 02:16:16.630352 25126 webserver.cc:460] Webserver started at http://127.15.75.130:42835/ using document root <none> and password file <none>
I20260809 02:16:16.630856 25126 fs_manager.cc:359] Metadata directory not provided
I20260809 02:16:16.630921 25126 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:16:16.631083 25126 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:16:16.633042 25126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-1/data/instance:
uuid: "6ee9ba5954274e238185dfb3cb465605"
format_stamp: "Formatted at 2026-08-09 02:16:16 on dist-test-slave-s11t"
server_key: "b3964c29242581d33be4f34fdff9da4c"
server_key_iv: "4dd5bb0c8836dcc1db1205f2494f0164"
server_key_version: "encryptionkey@0"
I20260809 02:16:16.633544 25126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-1/wal/instance:
uuid: "6ee9ba5954274e238185dfb3cb465605"
format_stamp: "Formatted at 2026-08-09 02:16:16 on dist-test-slave-s11t"
server_key: "b3964c29242581d33be4f34fdff9da4c"
server_key_iv: "4dd5bb0c8836dcc1db1205f2494f0164"
server_key_version: "encryptionkey@0"
I20260809 02:16:16.637048 25126 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 02:16:16.639281 25153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:16.640471 25126 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 02:16:16.640564 25126 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-1/wal
uuid: "6ee9ba5954274e238185dfb3cb465605"
format_stamp: "Formatted at 2026-08-09 02:16:16 on dist-test-slave-s11t"
server_key: "b3964c29242581d33be4f34fdff9da4c"
server_key_iv: "4dd5bb0c8836dcc1db1205f2494f0164"
server_key_version: "encryptionkey@0"
I20260809 02:16:16.640645 25126 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:16:16.658802 25126 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:16:16.662114 25126 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:16:16.662319 25126 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:16:16.662940 25126 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:16:16.663769 25126 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:16:16.663837 25126 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:16.663913 25126 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:16:16.663951 25126 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:16.673116 25126 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.130:36535
I20260809 02:16:16.673156 25324 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.130:36535 every 8 connection(s)
I20260809 02:16:16.674134 25126 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
I20260809 02:16:16.679682 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 25126
I20260809 02:16:16.679775 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-1/wal/instance
I20260809 02:16:16.680048 15662 external_mini_cluster.cc:1428] Setting key 99bc66030e0fabf911ced965f5d3f066
Aug 09 02:16:16 dist-test-slave-s11t krb5kdc[24745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241776, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.75.130@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM
I20260809 02:16:16.687695 24883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:16.675974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.130:60707 (local address 127.15.75.190:42501)
0809 02:16:16.676101 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:16.676105 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:16.676989 (+ 884us) server_negotiation.cc:407] Connection header received
0809 02:16:16.678268 (+ 1279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:16.678271 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:16.678325 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:16.678398 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:16.679493 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:16.680132 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:16.681436 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:16.681667 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:16.681772 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:16.684192 (+ 2420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:16.684195 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:16.684198 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:16.684228 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:16.685652 (+ 1424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:16.686153 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:16.686156 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:16.686158 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:16.686239 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:16.686569 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:16.686573 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:16.686576 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:16.686771 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:16.686897 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:16.687394 (+ 497us) server_negotiation.cc:299] Negotiation successful
0809 02:16:16.687520 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 02:16:16.688473 25328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:16.676346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:42501 (local address 127.15.75.130:60707)
0809 02:16:16.676838 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:16.676879 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:16.678088 (+ 1209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:16.678518 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:16.678524 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:16.678854 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:16.679325 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:16.679338 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:16.680277 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:16.680280 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:16.681298 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:16.681308 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:16.681824 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:16.681828 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:16.682003 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:16.682598 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:16.682614 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:16.684033 (+ 1419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:16:16.685770 (+ 1737us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:16.685776 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:16.685792 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:16.686050 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:16.686329 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:16.686333 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:16.686335 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:16.686474 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:16.686904 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:16:16.686912 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:16:16.687158 (+ 246us) client_negotiation.cc:769] Sending connection context
0809 02:16:16.687342 (+ 184us) client_negotiation.cc:240] Negotiation successful
0809 02:16:16.687656 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":118,"threads_started":1}
I20260809 02:16:16.689805 25325 heartbeater.cc:347] Connected to a master server at 127.15.75.190:42501
I20260809 02:16:16.690042 25325 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for oryx/127.15.75.131@KRBTEST.COM; defaulting to no policy
I20260809 02:16:16.690554 25325 heartbeater.cc:511] Master 127.15.75.190:42501 requested a full tablet report, sending...
I20260809 02:16:16.691601 24810 ts_manager.cc:194] Registered new tserver with Master: 6ee9ba5954274e238185dfb3cb465605 (127.15.75.130:36535)
I20260809 02:16:16.692163 24810 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.75.130@KRBTEST.COM'} at 127.15.75.130:60707
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.75.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.75.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/krb5kdc/oryx_127.15.75.131.keytab.
Entry for principal oryx/127.15.75.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/krb5kdc/oryx_127.15.75.131.keytab.
I20260809 02:16:16.707422 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.131:0
--local_ip_for_outbound_sockets=127.15.75.131
--webserver_interface=127.15.75.131
--webserver_port=0
--tserver_master_addrs=127.15.75.190:42501
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/krb5kdc/oryx_127.15.75.131.keytab
--principal=oryx/127.15.75.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:34317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:16:16.818689 25331 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:16:16.818958 25331 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:16:16.819025 25331 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:16:16.823212 25331 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:16:16.823287 25331 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:16:16.823370 25331 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.131
I20260809 02:16:16.828718 25331 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:34317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/krb5kdc/oryx_127.15.75.131.keytab
--principal=oryx/127.15.75.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.75.131
--webserver_port=0
--tserver_master_addrs=127.15.75.190:42501
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25331
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.131
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:16:16.829723 25331 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:16:16.830747 25331 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:16:16.840250 25339 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:16.840346 25340 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:16.840476 25331 server_base.cc:1034] running on GCE node
W20260809 02:16:16.840601 25344 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:16.841027 25331 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:16:16.841662 25331 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:16:16.842850 25331 hybrid_clock.cc:648] HybridClock initialized: now 1786241776842833 us; error 33 us; skew 500 ppm
Aug 09 02:16:16 dist-test-slave-s11t krb5kdc[24745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241776, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.75.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:16:16.845856 25331 init.cc:377] Logged in from keytab as oryx/127.15.75.131@KRBTEST.COM (short username oryx)
I20260809 02:16:16.846688 25331 webserver.cc:460] Webserver started at http://127.15.75.131:44281/ using document root <none> and password file <none>
I20260809 02:16:16.847214 25331 fs_manager.cc:359] Metadata directory not provided
I20260809 02:16:16.847275 25331 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:16:16.847440 25331 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:16:16.849521 25331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-2/data/instance:
uuid: "acf85e824204477bb6d89218e1ed0533"
format_stamp: "Formatted at 2026-08-09 02:16:16 on dist-test-slave-s11t"
server_key: "d338c3d724e036a0dbb3c1de2790e409"
server_key_iv: "7a7dd79b575d687e3076948f2e7aa024"
server_key_version: "encryptionkey@0"
I20260809 02:16:16.850054 25331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-2/wal/instance:
uuid: "acf85e824204477bb6d89218e1ed0533"
format_stamp: "Formatted at 2026-08-09 02:16:16 on dist-test-slave-s11t"
server_key: "d338c3d724e036a0dbb3c1de2790e409"
server_key_iv: "7a7dd79b575d687e3076948f2e7aa024"
server_key_version: "encryptionkey@0"
I20260809 02:16:16.853799 25331 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 02:16:16.856156 25354 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:16.857520 25331 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260809 02:16:16.857650 25331 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-2/wal
uuid: "acf85e824204477bb6d89218e1ed0533"
format_stamp: "Formatted at 2026-08-09 02:16:16 on dist-test-slave-s11t"
server_key: "d338c3d724e036a0dbb3c1de2790e409"
server_key_iv: "7a7dd79b575d687e3076948f2e7aa024"
server_key_version: "encryptionkey@0"
I20260809 02:16:16.857748 25331 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:16:16.878069 25331 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:16:16.881500 25331 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:16:16.881711 25331 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:16:16.882329 25331 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:16:16.883163 25331 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:16:16.883212 25331 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:16.883299 25331 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:16:16.883340 25331 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:16.892293 25331 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.131:35531
I20260809 02:16:16.892323 25519 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.131:35531 every 8 connection(s)
I20260809 02:16:16.893252 25331 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
I20260809 02:16:16.894025 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 25331
I20260809 02:16:16.894158 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-2/wal/instance
I20260809 02:16:16.894464 15662 external_mini_cluster.cc:1428] Setting key f912e9fd0eca1c8af199ebf40dbace23
Aug 09 02:16:16 dist-test-slave-s11t krb5kdc[24745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241776, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.75.131@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM
I20260809 02:16:16.906368 24883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:16.895313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.131:40585 (local address 127.15.75.190:42501)
0809 02:16:16.895462 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:16.895465 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:16.896177 (+ 712us) server_negotiation.cc:407] Connection header received
0809 02:16:16.897617 (+ 1440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:16.897620 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:16.897665 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:16.897740 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:16.898871 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:16.899407 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:16.900576 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:16.900858 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:16.900939 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:16.903372 (+ 2433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:16.903375 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:16.903378 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:16.903401 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:16.904616 (+ 1215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:16.905101 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:16.905103 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:16.905104 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:16.905179 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:16.905459 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:16.905461 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:16.905462 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:16.905628 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:16.905702 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:16.906140 (+ 438us) server_negotiation.cc:299] Negotiation successful
0809 02:16:16.906239 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 02:16:16.907096 25525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:16.895548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:42501 (local address 127.15.75.131:40585)
0809 02:16:16.896015 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:16.896079 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:16.897418 (+ 1339us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:16.897904 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:16.897910 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:16.898256 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:16.898712 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:16.898735 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:16.899560 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:16.899563 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:16.900467 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:16.900476 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:16.900997 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:16.900999 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:16.901142 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:16.901726 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:16.901743 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:16.903226 (+ 1483us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:16:16.904757 (+ 1531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:16.904761 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:16.904771 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:16.905017 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:16.905255 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:16.905257 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:16.905259 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:16.905381 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:16.905710 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:16:16.905715 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:16:16.905935 (+ 220us) client_negotiation.cc:769] Sending connection context
0809 02:16:16.906081 (+ 146us) client_negotiation.cc:240] Negotiation successful
0809 02:16:16.906311 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":135,"threads_started":1}
I20260809 02:16:16.908219 25522 heartbeater.cc:347] Connected to a master server at 127.15.75.190:42501
I20260809 02:16:16.908440 25522 heartbeater.cc:464] Registering TS with master...
I20260809 02:16:16.908923 25522 heartbeater.cc:511] Master 127.15.75.190:42501 requested a full tablet report, sending...
I20260809 02:16:16.909852 24808 ts_manager.cc:194] Registered new tserver with Master: acf85e824204477bb6d89218e1ed0533 (127.15.75.131:35531)
I20260809 02:16:16.910367 24808 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.75.131@KRBTEST.COM'} at 127.15.75.131:40585
I20260809 02:16:16.919905 15662 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.15.75.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.75.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.75.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/krb5kdc/oryx_127.15.75.189.keytab.
Entry for principal oryx/127.15.75.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/krb5kdc/oryx_127.15.75.189.keytab.
I20260809 02:16:16.948444 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.189:42709
--webserver_interface=127.15.75.189
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/krb5kdc/oryx_127.15.75.189.keytab
--principal=oryx/127.15.75.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:34317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.75.190:42501,127.15.75.189:42709
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:16:17.061456 25528 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:16:17.061712 25528 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:16:17.061779 25528 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:16:17.065773 25528 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:16:17.065852 25528 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:16:17.065893 25528 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:16:17.065946 25528 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:16:17.065986 25528 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:16:17.071080 25528 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:34317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.15.75.190:42501,127.15.75.189:42709
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/krb5kdc/oryx_127.15.75.189.keytab
--principal=oryx/127.15.75.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.189:42709
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.15.75.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25528
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
W20260809 02:16:17.071334 25528 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.15.75.190:42501,127.15.75.189:42709'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260809 02:16:17.072420 25528 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:16:17.073559 25528 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:16:17.082109 25544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:17.082120 25538 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:17.082198 25537 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:17.082301 25528 server_base.cc:1034] running on GCE node
I20260809 02:16:17.082978 25528 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:16:17.084007 25528 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:16:17.085186 25528 hybrid_clock.cc:648] HybridClock initialized: now 1786241777085154 us; error 45 us; skew 500 ppm
Aug 09 02:16:17 dist-test-slave-s11t krb5kdc[24745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241777, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.75.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:16:17.087900 25528 init.cc:377] Logged in from keytab as oryx/127.15.75.189@KRBTEST.COM (short username oryx)
I20260809 02:16:17.088899 25528 webserver.cc:460] Webserver started at http://127.15.75.189:38369/ using document root <none> and password file <none>
I20260809 02:16:17.089411 25528 fs_manager.cc:359] Metadata directory not provided
I20260809 02:16:17.089470 25528 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:16:17.089633 25528 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:16:17.091583 25528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/master-1/data/instance:
uuid: "b4f88c387e1d4b3c83ef38475e586f91"
format_stamp: "Formatted at 2026-08-09 02:16:17 on dist-test-slave-s11t"
server_key: "0272e9bc44cb65f66e889871d9d72d61"
server_key_iv: "d15b216d69944f290eef833296de7a90"
server_key_version: "encryptionkey@0"
I20260809 02:16:17.092094 25528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/master-1/wal/instance:
uuid: "b4f88c387e1d4b3c83ef38475e586f91"
format_stamp: "Formatted at 2026-08-09 02:16:17 on dist-test-slave-s11t"
server_key: "0272e9bc44cb65f66e889871d9d72d61"
server_key_iv: "d15b216d69944f290eef833296de7a90"
server_key_version: "encryptionkey@0"
I20260809 02:16:17.095749 25528 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260809 02:16:17.098063 25552 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:17.099263 25528 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 02:16:17.099401 25528 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/master-1/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/master-1/wal
uuid: "b4f88c387e1d4b3c83ef38475e586f91"
format_stamp: "Formatted at 2026-08-09 02:16:17 on dist-test-slave-s11t"
server_key: "0272e9bc44cb65f66e889871d9d72d61"
server_key_iv: "d15b216d69944f290eef833296de7a90"
server_key_version: "encryptionkey@0"
I20260809 02:16:17.099499 25528 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:16:17.116853 25528 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:16:17.119529 25528 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:16:17.119725 25528 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:16:17.127152 25528 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.189:42709
I20260809 02:16:17.127177 25634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.189:42709 every 8 connection(s)
I20260809 02:16:17.128145 25528 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/master-1/data/info.pb
I20260809 02:16:17.130779 25637 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:16:17.134377 25637 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.75.190" port: 42501 } has no permanent_uuid. Determining permanent_uuid...
I20260809 02:16:17.135461 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 25528
Aug 09 02:16:17 dist-test-slave-s11t krb5kdc[24745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241777, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.75.189@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM
Aug 09 02:16:17 dist-test-slave-s11t krb5kdc[24745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241775, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.75.189@KRBTEST.COM
I20260809 02:16:17.148811 24883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:17.136201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59528 (local address 127.15.75.190:42501)
0809 02:16:17.136336 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:17.136340 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:17.137472 (+ 1132us) server_negotiation.cc:407] Connection header received
0809 02:16:17.138659 (+ 1187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:17.138665 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:17.138746 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:17.138847 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:17.140046 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:17.140637 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:17.141902 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:17.142233 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:17.142343 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:17.145084 (+ 2741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:17.145088 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:17.145090 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:17.145120 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:17.146771 (+ 1651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:17.147292 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:17.147295 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:17.147297 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:17.147381 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:17.147709 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:17.147714 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:17.147716 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:17.147929 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:17.148035 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:17.148495 (+ 460us) server_negotiation.cc:299] Negotiation successful
0809 02:16:17.148610 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 02:16:17.149722 25640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:17.136647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:42501 (local address 127.0.0.1:59528)
0809 02:16:17.137327 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:17.137383 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:17.138355 (+ 972us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:17.139039 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:17.139045 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:17.139538 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:17.139892 (+ 354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:17.139907 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:17.140812 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:17.140816 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:17.141788 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:17.141796 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:17.142393 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:17.142397 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:17.142591 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:17.143210 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:17.143225 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:17.144943 (+ 1718us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:16:17.146883 (+ 1940us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:17.146894 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:17.146904 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:17.147183 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:17.147472 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:17.147476 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:17.147478 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:17.147615 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:17.148036 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:16:17.148043 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:16:17.148271 (+ 228us) client_negotiation.cc:769] Sending connection context
0809 02:16:17.148449 (+ 178us) client_negotiation.cc:240] Negotiation successful
0809 02:16:17.148754 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":145,"threads_started":1}
I20260809 02:16:17.149725 25641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:17.137622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41124 (local address 127.15.75.189:42709)
0809 02:16:17.137930 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:17.137942 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:17.137983 (+ 41us) server_negotiation.cc:407] Connection header received
0809 02:16:17.138355 (+ 372us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:17.138376 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:17.138631 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:17.138833 (+ 202us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:17.140181 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:17.141327 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:17.142286 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:17.142807 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:17.142927 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:17.145824 (+ 2897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:17.145831 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:17.145843 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:17.145879 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:17.147352 (+ 1473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:17.147797 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:17.147803 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:17.147804 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:17.147897 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:17.148179 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:17.148183 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:17.148185 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:17.148414 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:17.148615 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:17.149176 (+ 561us) server_negotiation.cc:299] Negotiation successful
0809 02:16:17.149312 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":96,"threads_started":1}
I20260809 02:16:17.154048 25637 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.75.189" port: 42709 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 02:16:17 dist-test-slave-s11t krb5kdc[24745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241777, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.75.189@KRBTEST.COM for oryx/127.15.75.189@KRBTEST.COM
I20260809 02:16:17.162694 25640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:17.154543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.189:42709 (local address 127.0.0.1:41128)
0809 02:16:17.154738 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:17.154751 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:17.154829 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:17.155050 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:17.155053 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:17.155198 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:17.155271 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:17.155276 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:17.155979 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:17.155983 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:17.156461 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:17.156468 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:17.157538 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:17.157541 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:17.157660 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:17.158099 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:17.158110 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:17.159452 (+ 1342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:16:17.161209 (+ 1757us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:17.161212 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:17.161214 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:17.161502 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:17.161780 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:17.161797 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:17.161798 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:17.161875 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:17.162217 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:16:17.162220 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:16:17.162315 (+ 95us) client_negotiation.cc:769] Sending connection context
0809 02:16:17.162420 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 02:16:17.162535 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":4}
I20260809 02:16:17.162710 25641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:17.154570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41128 (local address 127.15.75.189:42709)
0809 02:16:17.154661 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:17.154668 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:17.154908 (+ 240us) server_negotiation.cc:407] Connection header received
0809 02:16:17.154937 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:17.154939 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:17.154965 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:17.155038 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:17.155372 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:17.155871 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:17.156568 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:17.157426 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:17.157540 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:17.159576 (+ 2036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:17.159578 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:17.159580 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:17.159600 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:17.161110 (+ 1510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:17.161592 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:17.161594 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:17.161595 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:17.161692 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:17.161949 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:17.161951 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:17.161952 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:17.162101 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:17.162178 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:17.162469 (+ 291us) server_negotiation.cc:299] Negotiation successful
0809 02:16:17.162584 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":62,"server-negotiator.queue_time_us":31}
I20260809 02:16:17.165390 25637 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4f88c387e1d4b3c83ef38475e586f91: Bootstrap starting.
I20260809 02:16:17.167487 25637 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b4f88c387e1d4b3c83ef38475e586f91: Neither blocks nor log segments found. Creating new log.
I20260809 02:16:17.168183 25637 log.cc:824] T 00000000000000000000000000000000 P b4f88c387e1d4b3c83ef38475e586f91: Log is configured to *not* fsync() on all Append() calls
I20260809 02:16:17.170003 25637 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4f88c387e1d4b3c83ef38475e586f91: No bootstrap required, opened a new log
I20260809 02:16:17.172873 25637 raft_consensus.cc:348] T 00000000000000000000000000000000 P b4f88c387e1d4b3c83ef38475e586f91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59a0fdc35554648bd22e9b2280fe648" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 42501 } } peers { permanent_uuid: "b4f88c387e1d4b3c83ef38475e586f91" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 42709 } }
I20260809 02:16:17.173079 25637 raft_consensus.cc:374] T 00000000000000000000000000000000 P b4f88c387e1d4b3c83ef38475e586f91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:16:17.173221 25637 raft_consensus.cc:729] T 00000000000000000000000000000000 P b4f88c387e1d4b3c83ef38475e586f91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4f88c387e1d4b3c83ef38475e586f91, State: Initialized, Role: FOLLOWER
I20260809 02:16:17.173619 25637 consensus_queue.cc:260] T 00000000000000000000000000000000 P b4f88c387e1d4b3c83ef38475e586f91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59a0fdc35554648bd22e9b2280fe648" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 42501 } } peers { permanent_uuid: "b4f88c387e1d4b3c83ef38475e586f91" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 42709 } }
I20260809 02:16:17.174049 25644 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4f88c387e1d4b3c83ef38475e586f91 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59a0fdc35554648bd22e9b2280fe648" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 42501 } } peers { permanent_uuid: "b4f88c387e1d4b3c83ef38475e586f91" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 42709 } } }
I20260809 02:16:17.174166 25644 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4f88c387e1d4b3c83ef38475e586f91 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:16:17.174772 25637 sys_catalog.cc:564] T 00000000000000000000000000000000 P b4f88c387e1d4b3c83ef38475e586f91 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 02:16:17.179246 25665 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b4f88c387e1d4b3c83ef38475e586f91: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:16:17.179337 25665 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:16:17.182117 25528 master_runner.cc:428] Detected that this master b4f88c387e1d4b3c83ef38475e586f91 is joining an existing cluster
I20260809 02:16:17.182194 25528 master_runner.cc:432] Initiating AddMaster RPC to add 127.15.75.189:42709
I20260809 02:16:17.193744 24883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:17.184390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59542 (local address 127.15.75.190:42501)
0809 02:16:17.184521 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:17.184526 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:17.184829 (+ 303us) server_negotiation.cc:407] Connection header received
0809 02:16:17.185016 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:17.185019 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:17.185090 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:17.185172 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:17.185868 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:17.186689 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:17.187604 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:17.188032 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:17.188149 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:17.189778 (+ 1629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:17.189782 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:17.189785 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:17.189812 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:17.191707 (+ 1895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:17.192350 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:17.192357 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:17.192359 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:17.192461 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:17.192811 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:17.192815 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:17.192817 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:17.193008 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:17.193128 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:17.193479 (+ 351us) server_negotiation.cc:299] Negotiation successful
0809 02:16:17.193600 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 02:16:17.193773 25678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:17.184333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:42501 (local address 127.0.0.1:59542)
0809 02:16:17.184734 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:17.184752 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:17.184886 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:17.185313 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:17.185316 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:17.185530 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:17.185733 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:17.185751 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:17.186820 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:17.186822 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:17.187474 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:17.187482 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:17.188189 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:17.188194 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:17.188319 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:17.189096 (+ 777us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:17.189122 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:17.189637 (+ 515us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:16:17.191845 (+ 2208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:17.191852 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:17.191856 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:17.192195 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:17.192592 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:17.192596 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:17.192598 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:17.192725 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:17.193175 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:16:17.193178 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:16:17.193285 (+ 107us) client_negotiation.cc:769] Sending connection context
0809 02:16:17.193406 (+ 121us) client_negotiation.cc:240] Negotiation successful
0809 02:16:17.193556 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"mutex_wait_us":27,"thread_start_us":91,"threads_started":1}
I20260809 02:16:17.194252 25677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:17.184380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.189:42709 (local address 127.0.0.1:41130)
0809 02:16:17.184737 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:17.184762 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:17.184860 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:17.185133 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:17.185140 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:17.185479 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:17.185738 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:17.185745 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:17.187693 (+ 1948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:17.187697 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:17.188538 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:17.188552 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:17.189153 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:17.189157 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:17.189283 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:17.190080 (+ 797us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:17.190093 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:17.190587 (+ 494us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:16:17.192730 (+ 2143us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:17.192733 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:17.192735 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:17.193000 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:17.193313 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:17.193317 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:17.193319 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:17.193407 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:17.193778 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:16:17.193781 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:16:17.193848 (+ 67us) client_negotiation.cc:769] Sending connection context
0809 02:16:17.193961 (+ 113us) client_negotiation.cc:240] Negotiation successful
0809 02:16:17.194082 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"mutex_wait_us":43,"spinlock_wait_cycles":2304,"thread_start_us":105,"threads_started":1}
I20260809 02:16:17.194700 25641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:17.184747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41130 (local address 127.15.75.189:42709)
0809 02:16:17.184841 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:17.184845 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:17.184860 (+ 15us) server_negotiation.cc:407] Connection header received
0809 02:16:17.184965 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:17.184968 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:17.185012 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:17.185098 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:17.186833 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:17.187581 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:17.188667 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:17.189027 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:17.189130 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:17.190752 (+ 1622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:17.190755 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:17.190757 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:17.190777 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:17.192592 (+ 1815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:17.193111 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:17.193115 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:17.193117 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:17.193206 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:17.193501 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:17.193504 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:17.193506 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:17.193672 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:17.193748 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:17.194469 (+ 721us) server_negotiation.cc:299] Negotiation successful
0809 02:16:17.194566 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
Aug 09 02:16:17 dist-test-slave-s11t krb5kdc[24745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241776, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.75.190@KRBTEST.COM for oryx/127.15.75.189@KRBTEST.COM
I20260809 02:16:17.208995 25641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:17.199383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41146 (local address 127.15.75.189:42709)
0809 02:16:17.199480 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:17.199483 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:17.199494 (+ 11us) server_negotiation.cc:407] Connection header received
0809 02:16:17.199543 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:17.199545 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:17.199582 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:17.199646 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:17.200368 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:17.200952 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:17.202144 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:17.202365 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:17.202467 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:17.205111 (+ 2644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:17.205115 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:17.205118 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:17.205143 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:17.206848 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:17.207503 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:17.207506 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:17.207508 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:17.207591 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:17.207921 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:17.207924 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:17.207926 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:17.208113 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:17.208209 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:17.208665 (+ 456us) server_negotiation.cc:299] Negotiation successful
0809 02:16:17.208834 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 02:16:17.209012 25680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:17.198774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.189:42709 (local address 127.0.0.1:41146)
0809 02:16:17.199091 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:17.199131 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:17.199236 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:17.199711 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:17.199717 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:17.200130 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:17.200249 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:17.200256 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:17.201059 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:17.201062 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:17.202034 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:17.202045 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:17.202468 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:17.202472 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:17.202630 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:17.203173 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:17.203188 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:17.204962 (+ 1774us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:16:17.206983 (+ 2021us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:17.206997 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:17.207007 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:17.207384 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:17.207690 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:17.207696 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:17.207701 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:17.207816 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:17.208237 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:16:17.208256 (+ 19us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:16:17.208420 (+ 164us) client_negotiation.cc:769] Sending connection context
0809 02:16:17.208604 (+ 184us) client_negotiation.cc:240] Negotiation successful
0809 02:16:17.208839 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":91,"threads_started":1}
I20260809 02:16:17.210592 24808 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.15.75.189:42709: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "b4f88c387e1d4b3c83ef38475e586f91"
member_type: NON_VOTER
last_known_addr {
host: "127.15.75.189"
port: 42709
}
attrs {
promote: true
}
}
dest_uuid: "c59a0fdc35554648bd22e9b2280fe648"
cas_config_opid_index: -1
I20260809 02:16:17.211301 24808 consensus_queue.cc:237] T 00000000000000000000000000000000 P c59a0fdc35554648bd22e9b2280fe648 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c59a0fdc35554648bd22e9b2280fe648" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 42501 } } peers { permanent_uuid: "b4f88c387e1d4b3c83ef38475e586f91" member_type: NON_VOTER last_known_addr { host: "127.15.75.189" port: 42709 } attrs { promote: true } }
I20260809 02:16:17.213438 25686 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c59a0fdc35554648bd22e9b2280fe648 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER b4f88c387e1d4b3c83ef38475e586f91 (127.15.75.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c59a0fdc35554648bd22e9b2280fe648" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 42501 } } peers { permanent_uuid: "b4f88c387e1d4b3c83ef38475e586f91" member_type: NON_VOTER last_known_addr { host: "127.15.75.189" port: 42709 } attrs { promote: true } } }
I20260809 02:16:17.214126 25596 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b4f88c387e1d4b3c83ef38475e586f91 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:16:17.215039 25686 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.15.75.189:42709
I20260809 02:16:17.215500 25528 master.cc:537] Master@127.15.75.189:42709 shutting down...
I20260809 02:16:17.215521 25596 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b4f88c387e1d4b3c83ef38475e586f91 [term 1 FOLLOWER]: Refusing update from remote peer c59a0fdc35554648bd22e9b2280fe648: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260809 02:16:17.215423 25686 sys_catalog.cc:455] T 00000000000000000000000000000000 P c59a0fdc35554648bd22e9b2280fe648 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c59a0fdc35554648bd22e9b2280fe648" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c59a0fdc35554648bd22e9b2280fe648" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 42501 } } peers { permanent_uuid: "b4f88c387e1d4b3c83ef38475e586f91" member_type: NON_VOTER last_known_addr { host: "127.15.75.189" port: 42709 } attrs { promote: true } } }
I20260809 02:16:17.216934 25686 sys_catalog.cc:458] T 00000000000000000000000000000000 P c59a0fdc35554648bd22e9b2280fe648 [sys.catalog]: This master's current role is: LEADER
I20260809 02:16:17.217140 25686 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c59a0fdc35554648bd22e9b2280fe648 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f88c387e1d4b3c83ef38475e586f91" member_type: NON_VOTER last_known_addr { host: "127.15.75.189" port: 42709 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 02:16:17.219300 24790 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
WARNING: no policy specified for oryx/127.15.75.188@KRBTEST.COM; defaulting to no policy
I20260809 02:16:17.220583 25528 raft_consensus.cc:2229] T 00000000000000000000000000000000 P b4f88c387e1d4b3c83ef38475e586f91 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:16:17.220940 25528 raft_consensus.cc:2258] T 00000000000000000000000000000000 P b4f88c387e1d4b3c83ef38475e586f91 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:16:17.221076 25528 tablet_replica.cc:330] T 00000000000000000000000000000000 P b4f88c387e1d4b3c83ef38475e586f91: stopping tablet replica
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.75.188@KRBTEST.COM" created.
I20260809 02:16:17.228540 25528 master.cc:552] Master@127.15.75.189:42709 shutdown complete.
I20260809 02:16:17.228609 25528 master_runner.cc:305] Clearing existing system tablet
I20260809 02:16:17.228919 25528 fs_manager.cc:359] Metadata directory not provided
I20260809 02:16:17.228996 25528 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:16:17.229632 25680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:17.229282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.189:42709 (local address 127.0.0.1:41150)
0809 02:16:17.229467 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:17.229547 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.189:42709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
W20260809 02:16:17.229907 24790 consensus_peers.cc:487] T 00000000000000000000000000000000 P c59a0fdc35554648bd22e9b2280fe648 -> Peer b4f88c387e1d4b3c83ef38475e586f91 (127.15.75.189:42709): Couldn't send request to peer b4f88c387e1d4b3c83ef38475e586f91. Status: Network error: Client connection negotiation failed: client connection to 127.15.75.189:42709: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 02:16:17.230501 25528 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:16:17.231385 25697 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:17.231604 25528 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:16:17.231778 25528 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:16:17.231820 25528 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/master-1/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/master-1/wal
uuid: "b4f88c387e1d4b3c83ef38475e586f91"
format_stamp: "Formatted at 2026-08-09 02:16:17 on dist-test-slave-s11t"
server_key: "0272e9bc44cb65f66e889871d9d72d61"
server_key_iv: "d15b216d69944f290eef833296de7a90"
server_key_version: "encryptionkey@0"
I20260809 02:16:17.233732 25528 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P b4f88c387e1d4b3c83ef38475e586f91: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:16:17.236320 25528 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P b4f88c387e1d4b3c83ef38475e586f91: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 02:16:17.236430 25528 log.cc:1196] T 00000000000000000000000000000000 P b4f88c387e1d4b3c83ef38475e586f91: Deleting WAL directory at /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260809 02:16:17.236842 25528 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P b4f88c387e1d4b3c83ef38475e586f91: Deleting consensus metadata
I20260809 02:16:17.237039 25528 master_runner.cc:315] Copying system tablet from 127.15.75.190:42501
I20260809 02:16:17.238320 25528 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P b4f88c387e1d4b3c83ef38475e586f91: tablet copy: Beginning tablet copy session from remote peer at address 127.15.75.190:42501
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.75.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/krb5kdc/oryx_127.15.75.188.keytab.
Entry for principal oryx/127.15.75.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/krb5kdc/oryx_127.15.75.188.keytab.
I20260809 02:16:17.240453 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.188:40911
--webserver_interface=127.15.75.188
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/krb5kdc/oryx_127.15.75.188.keytab
--principal=oryx/127.15.75.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:34317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.75.190:42501,127.15.75.189:42709,127.15.75.188:40911
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 02:16:17.251955 24883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:17.238991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59544 (local address 127.15.75.190:42501)
0809 02:16:17.239128 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:17.239132 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:17.239417 (+ 285us) server_negotiation.cc:407] Connection header received
0809 02:16:17.239697 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:17.239704 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:17.239759 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:17.239819 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:17.242677 (+ 2858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:17.243283 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:17.245050 (+ 1767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:17.245523 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:17.245638 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:17.247384 (+ 1746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:17.247391 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:17.247396 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:17.247433 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:17.249462 (+ 2029us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:17.250094 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:17.250101 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:17.250106 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:17.250222 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:17.250649 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:17.250655 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:17.250659 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:17.250941 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:17.251037 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:17.251544 (+ 507us) server_negotiation.cc:299] Negotiation successful
0809 02:16:17.251701 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 02:16:17.252749 25706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:17.238909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:42501 (local address 127.0.0.1:59544)
0809 02:16:17.239313 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:17.239329 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:17.239555 (+ 226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:17.242255 (+ 2700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:17.242258 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:17.242429 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:17.242546 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:17.242552 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:17.243406 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:17.243411 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:17.244807 (+ 1396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:17.244824 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:17.245852 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:17.245860 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:17.246007 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:17.246613 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:17.246636 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:17.247173 (+ 537us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:16:17.249618 (+ 2445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:17.249625 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:17.249631 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:17.249946 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:17.250393 (+ 447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:17.250398 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:17.250402 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:17.250512 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:17.251137 (+ 625us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:16:17.251142 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:16:17.251292 (+ 150us) client_negotiation.cc:769] Sending connection context
0809 02:16:17.252207 (+ 915us) client_negotiation.cc:240] Negotiation successful
0809 02:16:17.252425 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":84,"threads_started":1}
I20260809 02:16:17.253439 24845 tablet_copy_service.cc:140] P c59a0fdc35554648bd22e9b2280fe648: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer b4f88c387e1d4b3c83ef38475e586f91 ({username='oryx', principal='oryx/127.15.75.189@KRBTEST.COM'} at 127.0.0.1:59544)
I20260809 02:16:17.253626 24845 tablet_copy_service.cc:161] P c59a0fdc35554648bd22e9b2280fe648: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer b4f88c387e1d4b3c83ef38475e586f91 at {username='oryx', principal='oryx/127.15.75.189@KRBTEST.COM'} at 127.0.0.1:59544: session id = b4f88c387e1d4b3c83ef38475e586f91-00000000000000000000000000000000
I20260809 02:16:17.255569 24845 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P c59a0fdc35554648bd22e9b2280fe648: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 02:16:17.257503 25528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 02:16:17.260118 25528 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P b4f88c387e1d4b3c83ef38475e586f91: tablet copy: Starting download of 0 data blocks...
I20260809 02:16:17.260334 25528 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P b4f88c387e1d4b3c83ef38475e586f91: tablet copy: Starting download of 1 WAL segments...
I20260809 02:16:17.262192 25528 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P b4f88c387e1d4b3c83ef38475e586f91: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 02:16:17.264204 24845 tablet_copy_service.cc:342] P c59a0fdc35554648bd22e9b2280fe648: Request end of tablet copy session b4f88c387e1d4b3c83ef38475e586f91-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.15.75.189@KRBTEST.COM'} at 127.0.0.1:59544
I20260809 02:16:17.264317 24845 tablet_copy_service.cc:433] P c59a0fdc35554648bd22e9b2280fe648: ending tablet copy session b4f88c387e1d4b3c83ef38475e586f91-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer b4f88c387e1d4b3c83ef38475e586f91
I20260809 02:16:17.272020 25528 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:16:17.273701 25710 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:17.273806 25711 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:17.277686 25716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:17.277748 25528 server_base.cc:1034] running on GCE node
I20260809 02:16:17.278075 25528 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:16:17.278512 25528 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:16:17.285880 25528 hybrid_clock.cc:648] HybridClock initialized: now 1786241777285855 us; error 43 us; skew 500 ppm
Aug 09 02:16:17 dist-test-slave-s11t krb5kdc[24745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241777, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.75.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:16:17.287611 25528 init.cc:377] Logged in from keytab as oryx/127.15.75.189@KRBTEST.COM (short username oryx)
I20260809 02:16:17.288100 25528 webserver.cc:460] Webserver started at http://127.15.75.189:44137/ using document root <none> and password file <none>
I20260809 02:16:17.288295 25528 fs_manager.cc:359] Metadata directory not provided
I20260809 02:16:17.288339 25528 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:16:17.289721 25528 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:16:17.290771 25729 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:17.291034 25528 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:16:17.291112 25528 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/master-1/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/master-1/wal
uuid: "b4f88c387e1d4b3c83ef38475e586f91"
format_stamp: "Formatted at 2026-08-09 02:16:17 on dist-test-slave-s11t"
server_key: "0272e9bc44cb65f66e889871d9d72d61"
server_key_iv: "d15b216d69944f290eef833296de7a90"
server_key_version: "encryptionkey@0"
I20260809 02:16:17.291194 25528 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:16:17.304579 25528 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:16:17.306815 25528 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:16:17.312192 25528 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.189:42709
I20260809 02:16:17.312227 25799 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.189:42709 every 8 connection(s)
I20260809 02:16:17.312816 25528 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/master-1/data/info.pb
I20260809 02:16:17.314807 25801 sys_catalog.cc:263] Verifying existing consensus state
I20260809 02:16:17.315913 25801 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4f88c387e1d4b3c83ef38475e586f91: Bootstrap starting.
I20260809 02:16:17.327620 25801 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4f88c387e1d4b3c83ef38475e586f91: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 02:16:17.327957 25801 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4f88c387e1d4b3c83ef38475e586f91: Bootstrap complete.
I20260809 02:16:17.328563 25801 raft_consensus.cc:348] T 00000000000000000000000000000000 P b4f88c387e1d4b3c83ef38475e586f91 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c59a0fdc35554648bd22e9b2280fe648" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 42501 } } peers { permanent_uuid: "b4f88c387e1d4b3c83ef38475e586f91" member_type: NON_VOTER last_known_addr { host: "127.15.75.189" port: 42709 } attrs { promote: true } }
I20260809 02:16:17.328737 25801 raft_consensus.cc:729] T 00000000000000000000000000000000 P b4f88c387e1d4b3c83ef38475e586f91 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b4f88c387e1d4b3c83ef38475e586f91, State: Initialized, Role: LEARNER
I20260809 02:16:17.328874 25801 consensus_queue.cc:260] T 00000000000000000000000000000000 P b4f88c387e1d4b3c83ef38475e586f91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c59a0fdc35554648bd22e9b2280fe648" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 42501 } } peers { permanent_uuid: "b4f88c387e1d4b3c83ef38475e586f91" member_type: NON_VOTER last_known_addr { host: "127.15.75.189" port: 42709 } attrs { promote: true } }
I20260809 02:16:17.329223 25806 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4f88c387e1d4b3c83ef38475e586f91 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c59a0fdc35554648bd22e9b2280fe648" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 42501 } } peers { permanent_uuid: "b4f88c387e1d4b3c83ef38475e586f91" member_type: NON_VOTER last_known_addr { host: "127.15.75.189" port: 42709 } attrs { promote: true } } }
I20260809 02:16:17.329399 25801 sys_catalog.cc:564] T 00000000000000000000000000000000 P b4f88c387e1d4b3c83ef38475e586f91 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:16:17.329406 25806 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4f88c387e1d4b3c83ef38475e586f91 [sys.catalog]: This master's current role is: LEARNER
I20260809 02:16:17.333092 25824 catalog_manager.cc:1259] Loaded cluster ID: 60284658a9354acb9a8cbfb97c102def
I20260809 02:16:17.333148 25824 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b4f88c387e1d4b3c83ef38475e586f91: loading cluster ID for follower catalog manager: success
I20260809 02:16:17.335081 25824 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b4f88c387e1d4b3c83ef38475e586f91: acquiring CA information for follower catalog manager: success
I20260809 02:16:17.336314 25824 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b4f88c387e1d4b3c83ef38475e586f91: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 02:16:17.385229 25707 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:16:17.385515 25707 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:16:17.385596 25707 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:16:17.389736 25707 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:16:17.389812 25707 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:16:17.389850 25707 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:16:17.389901 25707 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:16:17.389938 25707 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:16:17.395339 25707 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:34317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.15.75.190:42501,127.15.75.189:42709,127.15.75.188:40911
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/krb5kdc/oryx_127.15.75.188.keytab
--principal=oryx/127.15.75.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.188:40911
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.15.75.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25707
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:16:17.396353 25707 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:16:17.397411 25707 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:16:17.405567 25833 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:17.405562 25835 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:17.405580 25832 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:17.405843 25707 server_base.cc:1034] running on GCE node
I20260809 02:16:17.406458 25707 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:16:17.407473 25707 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:16:17.408671 25707 hybrid_clock.cc:648] HybridClock initialized: now 1786241777408640 us; error 41 us; skew 500 ppm
Aug 09 02:16:17 dist-test-slave-s11t krb5kdc[24745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241777, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.75.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:16:17.411474 25707 init.cc:377] Logged in from keytab as oryx/127.15.75.188@KRBTEST.COM (short username oryx)
I20260809 02:16:17.412325 25707 webserver.cc:460] Webserver started at http://127.15.75.188:44909/ using document root <none> and password file <none>
I20260809 02:16:17.412919 25707 fs_manager.cc:359] Metadata directory not provided
I20260809 02:16:17.412982 25707 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:16:17.413221 25707 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:16:17.415171 25707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/master-2/data/instance:
uuid: "a39abb470ce448908175ec3b2e32e19f"
format_stamp: "Formatted at 2026-08-09 02:16:17 on dist-test-slave-s11t"
server_key: "3c27309d81c2b0edc6b4340ecf75844c"
server_key_iv: "56ccbd5de8c24fd7abaffd79bebad585"
server_key_version: "encryptionkey@0"
I20260809 02:16:17.415747 25707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/master-2/wal/instance:
uuid: "a39abb470ce448908175ec3b2e32e19f"
format_stamp: "Formatted at 2026-08-09 02:16:17 on dist-test-slave-s11t"
server_key: "3c27309d81c2b0edc6b4340ecf75844c"
server_key_iv: "56ccbd5de8c24fd7abaffd79bebad585"
server_key_version: "encryptionkey@0"
I20260809 02:16:17.419404 25707 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260809 02:16:17.421705 25846 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:17.422932 25707 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 02:16:17.423069 25707 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/master-2/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/master-2/wal
uuid: "a39abb470ce448908175ec3b2e32e19f"
format_stamp: "Formatted at 2026-08-09 02:16:17 on dist-test-slave-s11t"
server_key: "3c27309d81c2b0edc6b4340ecf75844c"
server_key_iv: "56ccbd5de8c24fd7abaffd79bebad585"
server_key_version: "encryptionkey@0"
I20260809 02:16:17.423173 25707 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:16:17.443279 25707 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:16:17.446493 25707 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:16:17.446636 25707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:16:17.454234 25707 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.188:40911
I20260809 02:16:17.454541 25931 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.188:40911 every 8 connection(s)
I20260809 02:16:17.455312 25707 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/master-2/data/info.pb
I20260809 02:16:17.458140 25932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:16:17.458412 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 25707
I20260809 02:16:17.463254 25932 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.75.190" port: 42501 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 02:16:17 dist-test-slave-s11t krb5kdc[24745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241775, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.75.188@KRBTEST.COM
Aug 09 02:16:17 dist-test-slave-s11t krb5kdc[24745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241777, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.75.188@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM
I20260809 02:16:17.473526 25934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:17.459936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51374 (local address 127.15.75.188:40911)
0809 02:16:17.460361 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:17.460376 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:17.460412 (+ 36us) server_negotiation.cc:407] Connection header received
0809 02:16:17.461078 (+ 666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:17.461095 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:17.461469 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:17.461727 (+ 258us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:17.462669 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:17.463708 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:17.464848 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:17.465539 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:17.465706 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:17.468439 (+ 2733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:17.468444 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:17.468452 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:17.468476 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:17.470450 (+ 1974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:17.470953 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:17.470958 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:17.470962 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:17.471118 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:17.471432 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:17.471434 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:17.471435 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:17.471777 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:17.472000 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:17.472258 (+ 258us) server_negotiation.cc:299] Negotiation successful
0809 02:16:17.472547 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":131,"threads_started":1}
I20260809 02:16:17.475121 24883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:17.465236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59556 (local address 127.15.75.190:42501)
0809 02:16:17.465362 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:17.465365 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:17.465673 (+ 308us) server_negotiation.cc:407] Connection header received
0809 02:16:17.465903 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:17.465905 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:17.465949 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:17.466014 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:17.466900 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:17.467431 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:17.468460 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:17.468765 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:17.468895 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:17.471586 (+ 2691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:17.471589 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:17.471591 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:17.471616 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:17.473228 (+ 1612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:17.473725 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:17.473729 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:17.473731 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:17.473904 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:17.474194 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:17.474198 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:17.474200 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:17.474385 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:17.474494 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:17.474799 (+ 305us) server_negotiation.cc:299] Negotiation successful
0809 02:16:17.474921 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 02:16:17.475143 25935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:17.465082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:42501 (local address 127.0.0.1:59556)
0809 02:16:17.465542 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:17.465578 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:17.465771 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:17.466097 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:17.466101 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:17.466566 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:17.466757 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:17.466768 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:17.467561 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:17.467564 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:17.468328 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:17.468338 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:17.468903 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:17.468910 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:17.469065 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:17.469614 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:17.469634 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:17.471434 (+ 1800us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:16:17.473333 (+ 1899us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:17.473341 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:17.473345 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:17.473563 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:17.474023 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:17.474025 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:17.474027 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:17.474102 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:17.474493 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:16:17.474501 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:16:17.474608 (+ 107us) client_negotiation.cc:769] Sending connection context
0809 02:16:17.474741 (+ 133us) client_negotiation.cc:240] Negotiation successful
0809 02:16:17.474962 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":140,"threads_started":1}
I20260809 02:16:17.476320 25932 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.75.189" port: 42709 } has no permanent_uuid. Determining permanent_uuid...
I20260809 02:16:17.477694 25113 heartbeater.cc:503] Master 127.15.75.190:42501 was elected leader, sending a full tablet report...
Aug 09 02:16:17 dist-test-slave-s11t krb5kdc[24745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241777, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.75.188@KRBTEST.COM for oryx/127.15.75.189@KRBTEST.COM
I20260809 02:16:17.484123 25935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:17.476767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.189:42709 (local address 127.0.0.1:41154)
0809 02:16:17.476892 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:17.476904 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:17.476964 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:17.477355 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:17.477358 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:17.477538 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:17.477626 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:17.477632 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:17.478625 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:17.478630 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:17.479233 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:17.479243 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:17.479753 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:17.479757 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:17.479858 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:17.480284 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:17.480295 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:17.481465 (+ 1170us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:16:17.482964 (+ 1499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:17.482966 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:17.482967 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:17.483150 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:17.483390 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:17.483392 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:17.483393 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:17.483464 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:17.483771 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:16:17.483773 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:16:17.483841 (+ 68us) client_negotiation.cc:769] Sending connection context
0809 02:16:17.483932 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 02:16:17.484023 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260809 02:16:17.484246 25937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:17.476906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41154 (local address 127.15.75.189:42709)
0809 02:16:17.477154 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:17.477157 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:17.477169 (+ 12us) server_negotiation.cc:407] Connection header received
0809 02:16:17.477205 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:17.477207 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:17.477243 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:17.477340 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:17.477743 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:17.478520 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:17.479360 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:17.479588 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:17.479744 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:17.481591 (+ 1847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:17.481594 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:17.481596 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:17.481619 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:17.482863 (+ 1244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:17.483230 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:17.483232 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:17.483233 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:17.483310 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:17.483536 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:17.483538 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:17.483539 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:17.483697 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:17.483772 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:17.483986 (+ 214us) server_negotiation.cc:299] Negotiation successful
0809 02:16:17.484083 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":84,"threads_started":1}
I20260809 02:16:17.484890 25932 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.75.188" port: 40911 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 02:16:17 dist-test-slave-s11t krb5kdc[24745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241777, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.75.188@KRBTEST.COM for oryx/127.15.75.188@KRBTEST.COM
I20260809 02:16:17.492139 25935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:17.485323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.188:40911 (local address 127.0.0.1:51382)
0809 02:16:17.485454 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:17.485467 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:17.485549 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:17.485750 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:17.485752 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:17.485905 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:17.485961 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:17.485965 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:17.486645 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:17.486648 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:17.487123 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:17.487130 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:17.487560 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:17.487564 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:17.487636 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:17.487988 (+ 352us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:17.487998 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:17.489144 (+ 1146us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:16:17.490514 (+ 1370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:17.490516 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:17.490517 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:17.490697 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:17.490945 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:17.490947 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:17.490949 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:17.491043 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:17.491368 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:16:17.491371 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:16:17.491485 (+ 114us) client_negotiation.cc:769] Sending connection context
0809 02:16:17.491878 (+ 393us) client_negotiation.cc:240] Negotiation successful
0809 02:16:17.492016 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260809 02:16:17.492177 25934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:17.485374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51382 (local address 127.15.75.188:40911)
0809 02:16:17.485474 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:17.485480 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:17.485512 (+ 32us) server_negotiation.cc:407] Connection header received
0809 02:16:17.485627 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:17.485630 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:17.485665 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:17.485739 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:17.486072 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:17.486533 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:17.487231 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:17.487459 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:17.487562 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:17.489269 (+ 1707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:17.489272 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:17.489274 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:17.489294 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:17.490413 (+ 1119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:17.490774 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:17.490776 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:17.490777 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:17.490863 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:17.491117 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:17.491120 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:17.491121 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:17.491267 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:17.491355 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:17.491944 (+ 589us) server_negotiation.cc:299] Negotiation successful
0809 02:16:17.492063 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":49}
I20260809 02:16:17.495074 25932 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a39abb470ce448908175ec3b2e32e19f: Bootstrap starting.
I20260809 02:16:17.497467 25932 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a39abb470ce448908175ec3b2e32e19f: Neither blocks nor log segments found. Creating new log.
I20260809 02:16:17.498216 25932 log.cc:824] T 00000000000000000000000000000000 P a39abb470ce448908175ec3b2e32e19f: Log is configured to *not* fsync() on all Append() calls
I20260809 02:16:17.500090 25932 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a39abb470ce448908175ec3b2e32e19f: No bootstrap required, opened a new log
I20260809 02:16:17.503806 25932 raft_consensus.cc:348] T 00000000000000000000000000000000 P a39abb470ce448908175ec3b2e32e19f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59a0fdc35554648bd22e9b2280fe648" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 42501 } } peers { permanent_uuid: "b4f88c387e1d4b3c83ef38475e586f91" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 42709 } } peers { permanent_uuid: "a39abb470ce448908175ec3b2e32e19f" member_type: VOTER last_known_addr { host: "127.15.75.188" port: 40911 } }
I20260809 02:16:17.504005 25932 raft_consensus.cc:374] T 00000000000000000000000000000000 P a39abb470ce448908175ec3b2e32e19f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:16:17.504073 25932 raft_consensus.cc:729] T 00000000000000000000000000000000 P a39abb470ce448908175ec3b2e32e19f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a39abb470ce448908175ec3b2e32e19f, State: Initialized, Role: FOLLOWER
I20260809 02:16:17.504498 25932 consensus_queue.cc:260] T 00000000000000000000000000000000 P a39abb470ce448908175ec3b2e32e19f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59a0fdc35554648bd22e9b2280fe648" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 42501 } } peers { permanent_uuid: "b4f88c387e1d4b3c83ef38475e586f91" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 42709 } } peers { permanent_uuid: "a39abb470ce448908175ec3b2e32e19f" member_type: VOTER last_known_addr { host: "127.15.75.188" port: 40911 } }
I20260809 02:16:17.505030 25939 sys_catalog.cc:455] T 00000000000000000000000000000000 P a39abb470ce448908175ec3b2e32e19f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59a0fdc35554648bd22e9b2280fe648" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 42501 } } peers { permanent_uuid: "b4f88c387e1d4b3c83ef38475e586f91" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 42709 } } peers { permanent_uuid: "a39abb470ce448908175ec3b2e32e19f" member_type: VOTER last_known_addr { host: "127.15.75.188" port: 40911 } } }
I20260809 02:16:17.505170 25939 sys_catalog.cc:458] T 00000000000000000000000000000000 P a39abb470ce448908175ec3b2e32e19f [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:16:17.505733 25932 sys_catalog.cc:564] T 00000000000000000000000000000000 P a39abb470ce448908175ec3b2e32e19f [sys.catalog]: configured and running, proceeding with master startup.
W20260809 02:16:17.510495 25958 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a39abb470ce448908175ec3b2e32e19f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:16:17.510586 25958 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:16:17.514593 25707 master_runner.cc:428] Detected that this master a39abb470ce448908175ec3b2e32e19f is joining an existing cluster
I20260809 02:16:17.514647 25707 master_runner.cc:432] Initiating AddMaster RPC to add 127.15.75.188:40911
I20260809 02:16:17.526840 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 24916
I20260809 02:16:17.544148 25934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:17.517728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51388 (local address 127.15.75.188:40911)
0809 02:16:17.517877 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:17.517884 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:17.520986 (+ 3102us) server_negotiation.cc:407] Connection header received
0809 02:16:17.521125 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:17.521128 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:17.521163 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:17.521206 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:17.521806 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:17.522550 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:17.523691 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:17.523997 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:17.524065 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:17.525464 (+ 1399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:17.525468 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:17.525473 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:17.525508 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:17.527514 (+ 2006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:17.541588 (+ 14074us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:17.541594 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:17.541599 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:17.541682 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:17.542204 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:17.542210 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:17.542215 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:17.542475 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:17.542576 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:17.543717 (+ 1141us) server_negotiation.cc:299] Negotiation successful
0809 02:16:17.543867 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":65}
I20260809 02:16:17.544790 25966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:17.517801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.188:40911 (local address 127.0.0.1:51388)
0809 02:16:17.520900 (+ 3099us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:17.520915 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:17.521039 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:17.521347 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:17.521349 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:17.521567 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:17.521686 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:17.521694 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:17.522669 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:17.522672 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:17.523567 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:17.523578 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:17.524199 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:17.524203 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:17.524321 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:17.524883 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:17.524898 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:17.525300 (+ 402us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:16:17.534549 (+ 9249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:17.534552 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:17.534553 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:17.541483 (+ 6930us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:17.541781 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:17.541783 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:17.541786 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:17.541872 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:17.542627 (+ 755us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:16:17.542630 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:16:17.542773 (+ 143us) client_negotiation.cc:769] Sending connection context
0809 02:16:17.544330 (+ 1557us) client_negotiation.cc:240] Negotiation successful
0809 02:16:17.544497 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3009,"thread_start_us":102,"threads_started":1}
I20260809 02:16:17.545222 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.129:34105
--local_ip_for_outbound_sockets=127.15.75.129
--tserver_master_addrs=127.15.75.190:42501
--webserver_port=35407
--webserver_interface=127.15.75.129
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/krb5kdc/oryx_127.15.75.129.keytab
--principal=oryx/127.15.75.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:34317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.15.75.190:42501,127.15.75.189:42709
--tserver_master_addrs=127.15.75.190:42501,127.15.75.189:42709,127.15.75.188:40911 with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 02:16:17.545491 24883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:17.517507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59566 (local address 127.15.75.190:42501)
0809 02:16:17.517680 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:17.517685 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:17.518238 (+ 553us) server_negotiation.cc:407] Connection header received
0809 02:16:17.518383 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:17.518387 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:17.518445 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:17.518536 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:17.519331 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:17.520311 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:17.521828 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:17.522234 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:17.522320 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:17.523800 (+ 1480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:17.523804 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:17.523807 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:17.523840 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:17.525732 (+ 1892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:17.541231 (+ 15499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:17.541235 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:17.541237 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:17.541358 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:17.542926 (+ 1568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:17.542932 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:17.542937 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:17.543190 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:17.543273 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:17.545105 (+ 1832us) server_negotiation.cc:299] Negotiation successful
0809 02:16:17.545244 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260809 02:16:17.546172 25967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:17.517231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:42501 (local address 127.0.0.1:59566)
0809 02:16:17.518160 (+ 929us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:17.518173 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:17.518267 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:17.518590 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:17.518596 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:17.518938 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:17.519152 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:17.519164 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:17.520473 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:17.520479 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:17.521673 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:17.521685 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:17.522506 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:17.522510 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:17.522613 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:17.523199 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:17.523211 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:17.523665 (+ 454us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:16:17.525887 (+ 2222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:17.525894 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:17.525899 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:17.541121 (+ 15222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:17.541942 (+ 821us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:17.541947 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:17.541952 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:17.542084 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:17.543365 (+ 1281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:16:17.543371 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:16:17.543487 (+ 116us) client_negotiation.cc:769] Sending connection context
0809 02:16:17.545665 (+ 2178us) client_negotiation.cc:240] Negotiation successful
0809 02:16:17.545881 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":853,"mutex_wait_us":5,"spinlock_wait_cycles":48000,"thread_start_us":138,"threads_started":1}
I20260809 02:16:17.548177 25937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:17.517241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41166 (local address 127.15.75.189:42709)
0809 02:16:17.517392 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:17.517400 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:17.518181 (+ 781us) server_negotiation.cc:407] Connection header received
0809 02:16:17.518360 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:17.518368 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:17.518428 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:17.518476 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:17.519532 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:17.520421 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:17.522176 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:17.522556 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:17.522636 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:17.542174 (+ 19538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:17.542178 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:17.542181 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:17.542211 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:17.543863 (+ 1652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:17.546545 (+ 2682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:17.546551 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:17.546556 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:17.546660 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:17.547039 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:17.547044 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:17.547048 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:17.547322 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:17.547435 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:17.547816 (+ 381us) server_negotiation.cc:299] Negotiation successful
0809 02:16:17.547983 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 02:16:17.548877 25968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:17.517175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.189:42709 (local address 127.0.0.1:41166)
0809 02:16:17.518073 (+ 898us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:17.518090 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:17.518214 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:17.518722 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:17.518725 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:17.519033 (+ 308us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:17.519295 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:17.519308 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:17.520566 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:17.520568 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:17.522038 (+ 1470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:17.522048 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:17.522836 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:17.522842 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:17.522952 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:17.541518 (+ 18566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:17.541534 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:17.541993 (+ 459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:16:17.544006 (+ 2013us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:17.544010 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:17.544012 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:17.544779 (+ 767us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:17.546794 (+ 2015us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:17.546799 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:17.546803 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:17.546918 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:17.547504 (+ 586us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:16:17.547509 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:16:17.547609 (+ 100us) client_negotiation.cc:769] Sending connection context
0809 02:16:17.548409 (+ 800us) client_negotiation.cc:240] Negotiation successful
0809 02:16:17.548573 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":794,"mutex_wait_us":44,"thread_start_us":104,"threads_started":1}
Aug 09 02:16:17 dist-test-slave-s11t krb5kdc[24745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241776, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.75.190@KRBTEST.COM for oryx/127.15.75.188@KRBTEST.COM
I20260809 02:16:17.560523 25680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:17.551036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.188:40911 (local address 127.0.0.1:51390)
0809 02:16:17.551152 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:17.551169 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:17.551255 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:17.551558 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:17.551563 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:17.551782 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:17.551924 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:17.551935 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:17.552980 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:17.552987 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:17.554002 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:17.554017 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:17.554615 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:17.554622 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:17.554767 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:17.555263 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:17.555289 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:17.556969 (+ 1680us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:16:17.558971 (+ 2002us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:17.558974 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:17.558976 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:17.559230 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:17.559523 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:17.559527 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:17.559529 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:17.559618 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:17.560014 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:16:17.560017 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:16:17.560112 (+ 95us) client_negotiation.cc:769] Sending connection context
0809 02:16:17.560217 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 02:16:17.560361 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260809 02:16:17.560638 25934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:17.551073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51390 (local address 127.15.75.188:40911)
0809 02:16:17.551175 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:17.551179 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:17.551215 (+ 36us) server_negotiation.cc:407] Connection header received
0809 02:16:17.551373 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:17.551377 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:17.551427 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:17.551510 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:17.552052 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:17.552849 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:17.554135 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:17.554478 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:17.554582 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:17.557122 (+ 2540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:17.557128 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:17.557134 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:17.557166 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:17.558856 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:17.559335 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:17.559338 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:17.559340 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:17.559425 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:17.559711 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:17.559714 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:17.559716 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:17.559916 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:17.560013 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:17.560284 (+ 271us) server_negotiation.cc:299] Negotiation successful
0809 02:16:17.560400 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 02:16:17.561980 24808 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.15.75.188:40911: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "a39abb470ce448908175ec3b2e32e19f"
member_type: NON_VOTER
last_known_addr {
host: "127.15.75.188"
port: 40911
}
attrs {
promote: true
}
}
dest_uuid: "c59a0fdc35554648bd22e9b2280fe648"
cas_config_opid_index: 5
I20260809 02:16:17.562601 24808 consensus_queue.cc:237] T 00000000000000000000000000000000 P c59a0fdc35554648bd22e9b2280fe648 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c59a0fdc35554648bd22e9b2280fe648" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 42501 } } peers { permanent_uuid: "b4f88c387e1d4b3c83ef38475e586f91" member_type: NON_VOTER last_known_addr { host: "127.15.75.189" port: 42709 } attrs { promote: true } } peers { permanent_uuid: "a39abb470ce448908175ec3b2e32e19f" member_type: NON_VOTER last_known_addr { host: "127.15.75.188" port: 40911 } attrs { promote: true } }
I20260809 02:16:17.564016 25687 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c59a0fdc35554648bd22e9b2280fe648 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER a39abb470ce448908175ec3b2e32e19f (127.15.75.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c59a0fdc35554648bd22e9b2280fe648" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 42501 } } peers { permanent_uuid: "b4f88c387e1d4b3c83ef38475e586f91" member_type: NON_VOTER last_known_addr { host: "127.15.75.189" port: 42709 } attrs { promote: true } } peers { permanent_uuid: "a39abb470ce448908175ec3b2e32e19f" member_type: NON_VOTER last_known_addr { host: "127.15.75.188" port: 40911 } attrs { promote: true } } }
I20260809 02:16:17.565660 25687 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.15.75.188:40911
I20260809 02:16:17.565842 25687 sys_catalog.cc:455] T 00000000000000000000000000000000 P c59a0fdc35554648bd22e9b2280fe648 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c59a0fdc35554648bd22e9b2280fe648" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c59a0fdc35554648bd22e9b2280fe648" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 42501 } } peers { permanent_uuid: "b4f88c387e1d4b3c83ef38475e586f91" member_type: NON_VOTER last_known_addr { host: "127.15.75.189" port: 42709 } attrs { promote: true } } peers { permanent_uuid: "a39abb470ce448908175ec3b2e32e19f" member_type: NON_VOTER last_known_addr { host: "127.15.75.188" port: 40911 } attrs { promote: true } } }
I20260809 02:16:17.565935 25687 sys_catalog.cc:458] T 00000000000000000000000000000000 P c59a0fdc35554648bd22e9b2280fe648 [sys.catalog]: This master's current role is: LEADER
I20260809 02:16:17.566203 25707 master.cc:537] Master@127.15.75.188:40911 shutting down...
I20260809 02:16:17.566605 25877 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a39abb470ce448908175ec3b2e32e19f [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:16:17.568089 25877 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a39abb470ce448908175ec3b2e32e19f [term 1 FOLLOWER]: Refusing update from remote peer c59a0fdc35554648bd22e9b2280fe648: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 02:16:17.568768 25686 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c59a0fdc35554648bd22e9b2280fe648 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a39abb470ce448908175ec3b2e32e19f" member_type: NON_VOTER last_known_addr { host: "127.15.75.188" port: 40911 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 02:16:17.570657 24792 consensus_peers.cc:487] T 00000000000000000000000000000000 P c59a0fdc35554648bd22e9b2280fe648 -> Peer a39abb470ce448908175ec3b2e32e19f (127.15.75.188:40911): Couldn't send request to peer a39abb470ce448908175ec3b2e32e19f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260809 02:16:17.573356 25707 raft_consensus.cc:2229] T 00000000000000000000000000000000 P a39abb470ce448908175ec3b2e32e19f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:16:17.573675 25707 raft_consensus.cc:2258] T 00000000000000000000000000000000 P a39abb470ce448908175ec3b2e32e19f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:16:17.573784 25707 tablet_replica.cc:330] T 00000000000000000000000000000000 P a39abb470ce448908175ec3b2e32e19f: stopping tablet replica
I20260809 02:16:17.574144 25937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:17.563911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41178 (local address 127.15.75.189:42709)
0809 02:16:17.568820 (+ 4909us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:17.568824 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:17.568840 (+ 16us) server_negotiation.cc:407] Connection header received
0809 02:16:17.568891 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:17.568893 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:17.568952 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:17.569052 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:16:17.569495 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:17.570271 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:17.572153 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:17.572896 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:17.573074 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:17.573237 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:17.573836 (+ 599us) server_negotiation.cc:299] Negotiation successful
0809 02:16:17.573914 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4810}
I20260809 02:16:17.574537 25680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:17.563683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.189:42709 (local address 127.0.0.1:41178)
0809 02:16:17.563797 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:17.563811 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:17.563882 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:17.569203 (+ 5321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:17.569209 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:17.569232 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:16:17.569335 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:17.569344 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:17.570383 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:17.570389 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:17.572013 (+ 1624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:17.572029 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:17.573311 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:17.573315 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:17.573403 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:17.573522 (+ 119us) client_negotiation.cc:769] Sending connection context
0809 02:16:17.573600 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 02:16:17.573673 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260809 02:16:17.575188 25766 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b4f88c387e1d4b3c83ef38475e586f91 [term 1 LEARNER]: Refusing update from remote peer c59a0fdc35554648bd22e9b2280fe648: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 02:16:17.575603 25686 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c59a0fdc35554648bd22e9b2280fe648 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f88c387e1d4b3c83ef38475e586f91" member_type: NON_VOTER last_known_addr { host: "127.15.75.189" port: 42709 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 02:16:17.577447 25766 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b4f88c387e1d4b3c83ef38475e586f91 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER a39abb470ce448908175ec3b2e32e19f (127.15.75.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c59a0fdc35554648bd22e9b2280fe648" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 42501 } } peers { permanent_uuid: "b4f88c387e1d4b3c83ef38475e586f91" member_type: NON_VOTER last_known_addr { host: "127.15.75.189" port: 42709 } attrs { promote: true } } peers { permanent_uuid: "a39abb470ce448908175ec3b2e32e19f" member_type: NON_VOTER last_known_addr { host: "127.15.75.188" port: 40911 } attrs { promote: true } } }
I20260809 02:16:17.578951 25707 master.cc:552] Master@127.15.75.188:40911 shutdown complete.
I20260809 02:16:17.578990 25707 master_runner.cc:305] Clearing existing system tablet
I20260809 02:16:17.579222 25707 fs_manager.cc:359] Metadata directory not provided
I20260809 02:16:17.579267 25707 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:16:17.579674 25806 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4f88c387e1d4b3c83ef38475e586f91 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c59a0fdc35554648bd22e9b2280fe648. Latest consensus state: current_term: 1 leader_uuid: "c59a0fdc35554648bd22e9b2280fe648" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c59a0fdc35554648bd22e9b2280fe648" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 42501 } } peers { permanent_uuid: "b4f88c387e1d4b3c83ef38475e586f91" member_type: NON_VOTER last_known_addr { host: "127.15.75.189" port: 42709 } attrs { promote: true } } peers { permanent_uuid: "a39abb470ce448908175ec3b2e32e19f" member_type: NON_VOTER last_known_addr { host: "127.15.75.188" port: 40911 } attrs { promote: true } } }
I20260809 02:16:17.579784 25806 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4f88c387e1d4b3c83ef38475e586f91 [sys.catalog]: This master's current role is: LEARNER
I20260809 02:16:17.579874 25806 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4f88c387e1d4b3c83ef38475e586f91 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c59a0fdc35554648bd22e9b2280fe648" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c59a0fdc35554648bd22e9b2280fe648" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 42501 } } peers { permanent_uuid: "b4f88c387e1d4b3c83ef38475e586f91" member_type: NON_VOTER last_known_addr { host: "127.15.75.189" port: 42709 } attrs { promote: true } } peers { permanent_uuid: "a39abb470ce448908175ec3b2e32e19f" member_type: NON_VOTER last_known_addr { host: "127.15.75.188" port: 40911 } attrs { promote: true } } }
I20260809 02:16:17.579934 25806 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4f88c387e1d4b3c83ef38475e586f91 [sys.catalog]: This master's current role is: LEARNER
I20260809 02:16:17.580459 25707 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 02:16:17.581466 25687 raft_consensus.cc:1050] T 00000000000000000000000000000000 P c59a0fdc35554648bd22e9b2280fe648: attempting to promote NON_VOTER b4f88c387e1d4b3c83ef38475e586f91 to VOTER
I20260809 02:16:17.581884 25687 consensus_queue.cc:237] T 00000000000000000000000000000000 P c59a0fdc35554648bd22e9b2280fe648 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c59a0fdc35554648bd22e9b2280fe648" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 42501 } } peers { permanent_uuid: "b4f88c387e1d4b3c83ef38475e586f91" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 42709 } attrs { promote: false } } peers { permanent_uuid: "a39abb470ce448908175ec3b2e32e19f" member_type: NON_VOTER last_known_addr { host: "127.15.75.188" port: 40911 } attrs { promote: true } }
I20260809 02:16:17.583055 25766 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b4f88c387e1d4b3c83ef38475e586f91 [term 1 LEARNER]: Refusing update from remote peer c59a0fdc35554648bd22e9b2280fe648: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260809 02:16:17.583422 25680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:17.582937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.188:40911 (local address 127.0.0.1:51402)
0809 02:16:17.583302 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:17.583360 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.188:40911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":284}
I20260809 02:16:17.583760 25686 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c59a0fdc35554648bd22e9b2280fe648 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f88c387e1d4b3c83ef38475e586f91" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 42709 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260809 02:16:17.584153 25680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:17.583859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.188:40911 (local address 127.0.0.1:51410)
0809 02:16:17.584048 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:17.584094 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.188:40911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130}
W20260809 02:16:17.584335 24792 consensus_peers.cc:487] T 00000000000000000000000000000000 P c59a0fdc35554648bd22e9b2280fe648 -> Peer a39abb470ce448908175ec3b2e32e19f (127.15.75.188:40911): Couldn't send request to peer a39abb470ce448908175ec3b2e32e19f. Status: Network error: Client connection negotiation failed: client connection to 127.15.75.188:40911: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 02:16:17.585286 25687 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c59a0fdc35554648bd22e9b2280fe648 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, b4f88c387e1d4b3c83ef38475e586f91 (127.15.75.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c59a0fdc35554648bd22e9b2280fe648" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 42501 } } peers { permanent_uuid: "b4f88c387e1d4b3c83ef38475e586f91" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 42709 } attrs { promote: false } } peers { permanent_uuid: "a39abb470ce448908175ec3b2e32e19f" member_type: NON_VOTER last_known_addr { host: "127.15.75.188" port: 40911 } attrs { promote: true } } }
I20260809 02:16:17.586673 25686 sys_catalog.cc:455] T 00000000000000000000000000000000 P c59a0fdc35554648bd22e9b2280fe648 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c59a0fdc35554648bd22e9b2280fe648" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c59a0fdc35554648bd22e9b2280fe648" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 42501 } } peers { permanent_uuid: "b4f88c387e1d4b3c83ef38475e586f91" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 42709 } attrs { promote: false } } peers { permanent_uuid: "a39abb470ce448908175ec3b2e32e19f" member_type: NON_VOTER last_known_addr { host: "127.15.75.188" port: 40911 } attrs { promote: true } } }
I20260809 02:16:17.586783 25686 sys_catalog.cc:458] T 00000000000000000000000000000000 P c59a0fdc35554648bd22e9b2280fe648 [sys.catalog]: This master's current role is: LEADER
I20260809 02:16:17.587164 25687 sys_catalog.cc:455] T 00000000000000000000000000000000 P c59a0fdc35554648bd22e9b2280fe648 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c59a0fdc35554648bd22e9b2280fe648" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c59a0fdc35554648bd22e9b2280fe648" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 42501 } } peers { permanent_uuid: "b4f88c387e1d4b3c83ef38475e586f91" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 42709 } attrs { promote: false } } peers { permanent_uuid: "a39abb470ce448908175ec3b2e32e19f" member_type: NON_VOTER last_known_addr { host: "127.15.75.188" port: 40911 } attrs { promote: true } } }
I20260809 02:16:17.587265 25687 sys_catalog.cc:458] T 00000000000000000000000000000000 P c59a0fdc35554648bd22e9b2280fe648 [sys.catalog]: This master's current role is: LEADER
I20260809 02:16:17.587255 25766 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b4f88c387e1d4b3c83ef38475e586f91 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, b4f88c387e1d4b3c83ef38475e586f91 (127.15.75.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c59a0fdc35554648bd22e9b2280fe648" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 42501 } } peers { permanent_uuid: "b4f88c387e1d4b3c83ef38475e586f91" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 42709 } attrs { promote: false } } peers { permanent_uuid: "a39abb470ce448908175ec3b2e32e19f" member_type: NON_VOTER last_known_addr { host: "127.15.75.188" port: 40911 } attrs { promote: true } } }
I20260809 02:16:17.588537 25806 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4f88c387e1d4b3c83ef38475e586f91 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c59a0fdc35554648bd22e9b2280fe648" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c59a0fdc35554648bd22e9b2280fe648" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 42501 } } peers { permanent_uuid: "b4f88c387e1d4b3c83ef38475e586f91" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 42709 } attrs { promote: false } } peers { permanent_uuid: "a39abb470ce448908175ec3b2e32e19f" member_type: NON_VOTER last_known_addr { host: "127.15.75.188" port: 40911 } attrs { promote: true } } }
I20260809 02:16:17.588663 25806 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4f88c387e1d4b3c83ef38475e586f91 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:16:17.596856 25976 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:17.597121 25707 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:16:17.597257 25707 fs_manager.cc:725] Time spent opening block manager: real 0.016s user 0.000s sys 0.000s
I20260809 02:16:17.597293 25707 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/master-2/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/master-2/wal
uuid: "a39abb470ce448908175ec3b2e32e19f"
format_stamp: "Formatted at 2026-08-09 02:16:17 on dist-test-slave-s11t"
server_key: "3c27309d81c2b0edc6b4340ecf75844c"
server_key_iv: "56ccbd5de8c24fd7abaffd79bebad585"
server_key_version: "encryptionkey@0"
I20260809 02:16:17.599225 25707 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P a39abb470ce448908175ec3b2e32e19f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:16:17.601604 25707 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P a39abb470ce448908175ec3b2e32e19f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 02:16:17.601692 25707 log.cc:1196] T 00000000000000000000000000000000 P a39abb470ce448908175ec3b2e32e19f: Deleting WAL directory at /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260809 02:16:17.601940 25707 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P a39abb470ce448908175ec3b2e32e19f: Deleting consensus metadata
I20260809 02:16:17.602100 25707 master_runner.cc:315] Copying system tablet from 127.15.75.190:42501
I20260809 02:16:17.603243 25707 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P a39abb470ce448908175ec3b2e32e19f: tablet copy: Beginning tablet copy session from remote peer at address 127.15.75.190:42501
I20260809 02:16:17.612607 24883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:17.603833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59570 (local address 127.15.75.190:42501)
0809 02:16:17.603947 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:17.603952 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:17.604191 (+ 239us) server_negotiation.cc:407] Connection header received
0809 02:16:17.604336 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:17.604340 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:17.604390 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:17.604480 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:17.605030 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:17.605927 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:17.607129 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:17.607518 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:17.607620 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:17.608954 (+ 1334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:17.608958 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:17.608961 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:17.608987 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:17.610681 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:17.611229 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:17.611233 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:17.611235 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:17.611334 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:17.611694 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:17.611698 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:17.611699 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:17.611914 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:17.612015 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:17.612312 (+ 297us) server_negotiation.cc:299] Negotiation successful
0809 02:16:17.612427 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 02:16:17.612609 25983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:17.603760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:42501 (local address 127.0.0.1:59570)
0809 02:16:17.604094 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:17.604108 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:17.604223 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:17.604514 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:17.604517 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:17.604766 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:17.604903 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:17.604911 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:17.606038 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:17.606042 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:17.606999 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:17.607010 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:17.607628 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:17.607632 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:17.607727 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:17.608334 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:17.608349 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:17.608799 (+ 450us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:16:17.610799 (+ 2000us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:17.610803 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:17.610805 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:17.611107 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:17.611497 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:17.611500 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:17.611502 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:17.611595 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:17.612020 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:16:17.612023 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:16:17.612124 (+ 101us) client_negotiation.cc:769] Sending connection context
0809 02:16:17.612240 (+ 116us) client_negotiation.cc:240] Negotiation successful
0809 02:16:17.612401 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":97,"threads_started":1}
I20260809 02:16:17.614089 24845 tablet_copy_service.cc:140] P c59a0fdc35554648bd22e9b2280fe648: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer a39abb470ce448908175ec3b2e32e19f ({username='oryx', principal='oryx/127.15.75.188@KRBTEST.COM'} at 127.0.0.1:59570)
I20260809 02:16:17.614189 24845 tablet_copy_service.cc:161] P c59a0fdc35554648bd22e9b2280fe648: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer a39abb470ce448908175ec3b2e32e19f at {username='oryx', principal='oryx/127.15.75.188@KRBTEST.COM'} at 127.0.0.1:59570: session id = a39abb470ce448908175ec3b2e32e19f-00000000000000000000000000000000
I20260809 02:16:17.615275 24845 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P c59a0fdc35554648bd22e9b2280fe648: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 02:16:17.616518 25707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 02:16:17.618804 25707 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P a39abb470ce448908175ec3b2e32e19f: tablet copy: Starting download of 0 data blocks...
I20260809 02:16:17.618995 25707 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P a39abb470ce448908175ec3b2e32e19f: tablet copy: Starting download of 1 WAL segments...
I20260809 02:16:17.620431 25707 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P a39abb470ce448908175ec3b2e32e19f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 02:16:17.622227 24845 tablet_copy_service.cc:342] P c59a0fdc35554648bd22e9b2280fe648: Request end of tablet copy session a39abb470ce448908175ec3b2e32e19f-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.15.75.188@KRBTEST.COM'} at 127.0.0.1:59570
I20260809 02:16:17.622314 24845 tablet_copy_service.cc:433] P c59a0fdc35554648bd22e9b2280fe648: ending tablet copy session a39abb470ce448908175ec3b2e32e19f-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer a39abb470ce448908175ec3b2e32e19f
I20260809 02:16:17.629966 25707 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:16:17.631572 25985 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:17.633466 25987 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:17.633796 25707 server_base.cc:1034] running on GCE node
W20260809 02:16:17.633987 25989 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:17.634239 25707 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:16:17.634522 25707 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:16:17.635656 25707 hybrid_clock.cc:648] HybridClock initialized: now 1786241777635645 us; error 28 us; skew 500 ppm
Aug 09 02:16:17 dist-test-slave-s11t krb5kdc[24745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241777, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.75.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:16:17.637327 25707 init.cc:377] Logged in from keytab as oryx/127.15.75.188@KRBTEST.COM (short username oryx)
I20260809 02:16:17.637748 25707 webserver.cc:460] Webserver started at http://127.15.75.188:39037/ using document root <none> and password file <none>
I20260809 02:16:17.637933 25707 fs_manager.cc:359] Metadata directory not provided
I20260809 02:16:17.637974 25707 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:16:17.639418 25707 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:16:17.640445 26003 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:17.640746 25707 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:16:17.640823 25707 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/master-2/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/master-2/wal
uuid: "a39abb470ce448908175ec3b2e32e19f"
format_stamp: "Formatted at 2026-08-09 02:16:17 on dist-test-slave-s11t"
server_key: "3c27309d81c2b0edc6b4340ecf75844c"
server_key_iv: "56ccbd5de8c24fd7abaffd79bebad585"
server_key_version: "encryptionkey@0"
I20260809 02:16:17.640908 25707 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:16:17.649646 25707 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:16:17.651759 25707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:16:17.657217 25707 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.188:40911
I20260809 02:16:17.657318 26078 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.188:40911 every 8 connection(s)
I20260809 02:16:17.657809 25707 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/master-2/data/info.pb
I20260809 02:16:17.659739 26079 sys_catalog.cc:263] Verifying existing consensus state
I20260809 02:16:17.660851 26079 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a39abb470ce448908175ec3b2e32e19f: Bootstrap starting.
I20260809 02:16:17.672528 26079 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a39abb470ce448908175ec3b2e32e19f: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 02:16:17.672873 26079 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a39abb470ce448908175ec3b2e32e19f: Bootstrap complete.
I20260809 02:16:17.673523 26079 raft_consensus.cc:348] T 00000000000000000000000000000000 P a39abb470ce448908175ec3b2e32e19f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c59a0fdc35554648bd22e9b2280fe648" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 42501 } } peers { permanent_uuid: "b4f88c387e1d4b3c83ef38475e586f91" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 42709 } attrs { promote: false } } peers { permanent_uuid: "a39abb470ce448908175ec3b2e32e19f" member_type: NON_VOTER last_known_addr { host: "127.15.75.188" port: 40911 } attrs { promote: true } }
I20260809 02:16:17.673676 26079 raft_consensus.cc:729] T 00000000000000000000000000000000 P a39abb470ce448908175ec3b2e32e19f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a39abb470ce448908175ec3b2e32e19f, State: Initialized, Role: LEARNER
I20260809 02:16:17.673795 26079 consensus_queue.cc:260] T 00000000000000000000000000000000 P a39abb470ce448908175ec3b2e32e19f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c59a0fdc35554648bd22e9b2280fe648" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 42501 } } peers { permanent_uuid: "b4f88c387e1d4b3c83ef38475e586f91" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 42709 } attrs { promote: false } } peers { permanent_uuid: "a39abb470ce448908175ec3b2e32e19f" member_type: NON_VOTER last_known_addr { host: "127.15.75.188" port: 40911 } attrs { promote: true } }
I20260809 02:16:17.674237 26079 sys_catalog.cc:564] T 00000000000000000000000000000000 P a39abb470ce448908175ec3b2e32e19f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:16:17.678910 26101 catalog_manager.cc:1259] Loaded cluster ID: 60284658a9354acb9a8cbfb97c102def
I20260809 02:16:17.678968 26101 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a39abb470ce448908175ec3b2e32e19f: loading cluster ID for follower catalog manager: success
I20260809 02:16:17.680413 26101 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a39abb470ce448908175ec3b2e32e19f: acquiring CA information for follower catalog manager: success
I20260809 02:16:17.680811 26084 sys_catalog.cc:455] T 00000000000000000000000000000000 P a39abb470ce448908175ec3b2e32e19f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c59a0fdc35554648bd22e9b2280fe648" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 42501 } } peers { permanent_uuid: "b4f88c387e1d4b3c83ef38475e586f91" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 42709 } attrs { promote: false } } peers { permanent_uuid: "a39abb470ce448908175ec3b2e32e19f" member_type: NON_VOTER last_known_addr { host: "127.15.75.188" port: 40911 } attrs { promote: true } } }
I20260809 02:16:17.681012 26084 sys_catalog.cc:458] T 00000000000000000000000000000000 P a39abb470ce448908175ec3b2e32e19f [sys.catalog]: This master's current role is: LEARNER
I20260809 02:16:17.681659 26101 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a39abb470ce448908175ec3b2e32e19f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 02:16:17.694315 25325 heartbeater.cc:503] Master 127.15.75.190:42501 was elected leader, sending a full tablet report...
W20260809 02:16:17.695663 25969 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:16:17.695964 25969 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:16:17.696025 25969 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:16:17.702030 25969 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:16:17.702095 25969 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:16:17.702188 25969 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.129
I20260809 02:16:17.710175 25969 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:34317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/krb5kdc/oryx_127.15.75.129.keytab
--principal=oryx/127.15.75.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.129:34105
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.75.129
--webserver_port=35407
--tserver_master_addrs=127.15.75.190:42501,127.15.75.189:42709,127.15.75.188:40911
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25969
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.129
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:16:17.711382 25969 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:16:17.712622 25969 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:16:17.723630 26116 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:17.723686 26110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:17.723729 25969 server_base.cc:1034] running on GCE node
W20260809 02:16:17.723973 26112 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:17.724361 25969 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:16:17.725111 25969 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:16:17.726306 25969 hybrid_clock.cc:648] HybridClock initialized: now 1786241777726264 us; error 57 us; skew 500 ppm
Aug 09 02:16:17 dist-test-slave-s11t krb5kdc[24745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241777, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.75.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:16:17.729242 25969 init.cc:377] Logged in from keytab as oryx/127.15.75.129@KRBTEST.COM (short username oryx)
I20260809 02:16:17.730176 25969 webserver.cc:460] Webserver started at http://127.15.75.129:35407/ using document root <none> and password file <none>
I20260809 02:16:17.730774 25969 fs_manager.cc:359] Metadata directory not provided
I20260809 02:16:17.730866 25969 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:16:17.734694 25969 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260809 02:16:17.736662 26131 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:17.737931 25969 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 02:16:17.738052 25969 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-0/wal
uuid: "6babbe94e6d54368a5c7edf6c5847ba1"
format_stamp: "Formatted at 2026-08-09 02:16:16 on dist-test-slave-s11t"
server_key: "556b0a5c7d4e5e06bd12aa213a0f1405"
server_key_iv: "85a32203d1f6bf9343c64eec7351f41f"
server_key_version: "encryptionkey@0"
I20260809 02:16:17.738566 25969 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:16:17.754072 25969 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:16:17.757159 25969 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:16:17.757357 25969 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:16:17.758585 25969 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:16:17.759411 25969 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:16:17.759459 25969 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:17.759524 25969 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:16:17.759557 25969 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:17.768640 25969 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.129:34105
I20260809 02:16:17.768726 26308 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.129:34105 every 8 connection(s)
I20260809 02:16:17.769619 25969 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
I20260809 02:16:17.773990 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 25969
I20260809 02:16:17.774225 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 25126
Aug 09 02:16:17 dist-test-slave-s11t krb5kdc[24745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241777, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.75.129@KRBTEST.COM for oryx/127.15.75.188@KRBTEST.COM
Aug 09 02:16:17 dist-test-slave-s11t krb5kdc[24745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241777, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.75.129@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM
Aug 09 02:16:17 dist-test-slave-s11t krb5kdc[24745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241777, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.75.129@KRBTEST.COM for oryx/127.15.75.189@KRBTEST.COM
I20260809 02:16:17.797540 24883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:17.771387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.129:38797 (local address 127.15.75.190:42501)
0809 02:16:17.771551 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:17.771556 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:17.772726 (+ 1170us) server_negotiation.cc:407] Connection header received
0809 02:16:17.774019 (+ 1293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:17.774023 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:17.774097 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:17.774154 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:17.775710 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:17.776859 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:17.780851 (+ 3992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:17.781338 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:17.781454 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:17.792163 (+ 10709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:17.792170 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:17.792177 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:17.792214 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:17.794967 (+ 2753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:17.795726 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:17.795730 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:17.795731 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:17.795825 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:17.796173 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:17.796176 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:17.796177 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:17.796393 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:17.796483 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:17.797230 (+ 747us) server_negotiation.cc:299] Negotiation successful
0809 02:16:17.797362 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 02:16:17.797712 25937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:17.771493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.129:60207 (local address 127.15.75.189:42709)
0809 02:16:17.771610 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:17.771613 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:17.772720 (+ 1107us) server_negotiation.cc:407] Connection header received
0809 02:16:17.774101 (+ 1381us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:17.774105 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:17.774159 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:17.774226 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:17.783440 (+ 9214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:17.784264 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:17.785383 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:17.785739 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:17.785831 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:17.792897 (+ 7066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:17.792903 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:17.792909 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:17.792945 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:17.794796 (+ 1851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:17.795626 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:17.795632 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:17.795637 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:17.795728 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:17.796137 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:17.796143 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:17.796147 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:17.796421 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:17.796525 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:17.797361 (+ 836us) server_negotiation.cc:299] Negotiation successful
0809 02:16:17.797506 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 02:16:17.798444 26321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:17.773133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.129:60699 (local address 127.15.75.188:40911)
0809 02:16:17.773368 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:17.773372 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:17.773386 (+ 14us) server_negotiation.cc:407] Connection header received
0809 02:16:17.774106 (+ 720us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:17.774111 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:17.774177 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:17.774266 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:17.776087 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:17.777236 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:17.780202 (+ 2966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:17.780550 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:17.780653 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:17.793308 (+ 12655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:17.793315 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:17.793318 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:17.793351 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:17.795134 (+ 1783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:17.795716 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:17.795719 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:17.795721 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:17.795825 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:17.796170 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:17.796173 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:17.796175 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:17.796393 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:17.796483 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:17.798154 (+ 1671us) server_negotiation.cc:299] Negotiation successful
0809 02:16:17.798295 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":90,"threads_started":1}
I20260809 02:16:17.798894 26317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:17.771909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.189:42709 (local address 127.15.75.129:60207)
0809 02:16:17.772539 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:17.772598 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:17.773759 (+ 1161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:17.782303 (+ 8544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:17.782309 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:17.783158 (+ 849us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:17.783282 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:17.783290 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:17.784397 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:17.784402 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:17.785251 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:17.785264 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:17.785973 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:17.785979 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:17.786076 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:17.786677 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:17.786702 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:17.792749 (+ 6047us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:16:17.794937 (+ 2188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:17.794946 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:17.794966 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:17.795506 (+ 540us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:17.795829 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:17.795835 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:17.795840 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:17.796025 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:17.796577 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:16:17.796583 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:16:17.796928 (+ 345us) client_negotiation.cc:769] Sending connection context
0809 02:16:17.797138 (+ 210us) client_negotiation.cc:240] Negotiation successful
0809 02:16:17.797483 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"mutex_wait_us":55,"spinlock_wait_cycles":14976,"thread_start_us":186,"threads_started":1}
I20260809 02:16:17.799180 26318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:17.771909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:42501 (local address 127.15.75.129:38797)
0809 02:16:17.772539 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:17.772598 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:17.773759 (+ 1161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:17.774459 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:17.774469 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:17.774960 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:17.775540 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:17.775558 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:17.777433 (+ 1875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:17.777440 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:17.780631 (+ 3191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:17.780647 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:17.781659 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:17.781667 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:17.781839 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:17.783172 (+ 1333us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:17.783188 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:17.791931 (+ 8743us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:16:17.795106 (+ 3175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:17.795109 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:17.795111 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:17.795598 (+ 487us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:17.795938 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:17.795940 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:17.795942 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:17.796061 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:17.796542 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:16:17.796549 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:16:17.796942 (+ 393us) client_negotiation.cc:769] Sending connection context
0809 02:16:17.797724 (+ 782us) client_negotiation.cc:240] Negotiation successful
0809 02:16:17.797862 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"mutex_wait_us":2040,"spinlock_wait_cycles":61824,"thread_start_us":145,"threads_started":1}
I20260809 02:16:17.799875 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.130:36535
--local_ip_for_outbound_sockets=127.15.75.130
--tserver_master_addrs=127.15.75.190:42501
--webserver_port=42835
--webserver_interface=127.15.75.130
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/krb5kdc/oryx_127.15.75.130.keytab
--principal=oryx/127.15.75.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:34317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.15.75.190:42501,127.15.75.189:42709
--tserver_master_addrs=127.15.75.190:42501,127.15.75.189:42709,127.15.75.188:40911 with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 02:16:17.800161 26313 heartbeater.cc:347] Connected to a master server at 127.15.75.189:42709
I20260809 02:16:17.800413 26313 heartbeater.cc:464] Registering TS with master...
I20260809 02:16:17.800757 26312 heartbeater.cc:347] Connected to a master server at 127.15.75.190:42501
I20260809 02:16:17.800846 26312 heartbeater.cc:464] Registering TS with master...
I20260809 02:16:17.800984 26313 heartbeater.cc:511] Master 127.15.75.189:42709 requested a full tablet report, sending...
I20260809 02:16:17.800994 26312 heartbeater.cc:511] Master 127.15.75.190:42501 requested a full tablet report, sending...
I20260809 02:16:17.801276 26319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:17.773025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.188:40911 (local address 127.15.75.129:60699)
0809 02:16:17.773297 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:17.773311 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:17.773773 (+ 462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:17.774465 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:17.774472 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:17.775825 (+ 1353us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:17.775960 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:17.775969 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:17.777393 (+ 1424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:17.777399 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:17.779294 (+ 1895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:17.779310 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:17.780834 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:17.780839 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:17.781053 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:17.782345 (+ 1292us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:17.782369 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:17.793123 (+ 10754us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:16:17.795265 (+ 2142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:17.795270 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:17.795272 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:17.795598 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:17.795937 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:17.795940 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:17.795942 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:17.796068 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:17.796542 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:16:17.796549 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:16:17.796942 (+ 393us) client_negotiation.cc:769] Sending connection context
0809 02:16:17.797138 (+ 196us) client_negotiation.cc:240] Negotiation successful
0809 02:16:17.798953 (+ 1815us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"mutex_wait_us":49,"spinlock_wait_cycles":6272,"thread_start_us":101,"threads_started":1}
I20260809 02:16:17.801944 26309 heartbeater.cc:347] Connected to a master server at 127.15.75.188:40911
I20260809 02:16:17.802023 26309 heartbeater.cc:464] Registering TS with master...
I20260809 02:16:17.802171 26309 heartbeater.cc:511] Master 127.15.75.188:40911 requested a full tablet report, sending...
I20260809 02:16:17.802357 24808 ts_manager.cc:194] Re-registered known tserver with Master: 6babbe94e6d54368a5c7edf6c5847ba1 (127.15.75.129:34105)
I20260809 02:16:17.802619 25749 ts_manager.cc:194] Registered new tserver with Master: 6babbe94e6d54368a5c7edf6c5847ba1 (127.15.75.129:34105)
I20260809 02:16:17.803184 26021 ts_manager.cc:194] Registered new tserver with Master: 6babbe94e6d54368a5c7edf6c5847ba1 (127.15.75.129:34105)
I20260809 02:16:17.803207 24808 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.75.129@KRBTEST.COM'} at 127.15.75.129:38797
I20260809 02:16:17.912144 25522 heartbeater.cc:503] Master 127.15.75.190:42501 was elected leader, sending a full tablet report...
W20260809 02:16:17.915733 26323 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:16:17.916014 26323 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:16:17.916101 26323 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:16:17.920871 26323 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:16:17.920944 26323 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:16:17.921046 26323 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.130
I20260809 02:16:17.926914 26323 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:34317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/krb5kdc/oryx_127.15.75.130.keytab
--principal=oryx/127.15.75.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.130:36535
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.75.130
--webserver_port=42835
--tserver_master_addrs=127.15.75.190:42501,127.15.75.189:42709,127.15.75.188:40911
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26323
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.130
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:16:17.927984 26323 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:16:17.929082 26323 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 02:16:17.939599 26323 server_base.cc:1034] running on GCE node
W20260809 02:16:17.939504 26332 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:17.939517 26330 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:17.939808 26328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:17.940217 26323 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:16:17.940913 26323 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:16:17.942078 26323 hybrid_clock.cc:648] HybridClock initialized: now 1786241777942043 us; error 42 us; skew 500 ppm
Aug 09 02:16:17 dist-test-slave-s11t krb5kdc[24745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241777, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.75.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:16:17.945338 26323 init.cc:377] Logged in from keytab as oryx/127.15.75.130@KRBTEST.COM (short username oryx)
I20260809 02:16:17.946171 26323 webserver.cc:460] Webserver started at http://127.15.75.130:42835/ using document root <none> and password file <none>
I20260809 02:16:17.946712 26323 fs_manager.cc:359] Metadata directory not provided
I20260809 02:16:17.946787 26323 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:16:17.950546 26323 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260809 02:16:17.952495 26340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:17.953729 26323 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 02:16:17.953866 26323 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-1/wal
uuid: "6ee9ba5954274e238185dfb3cb465605"
format_stamp: "Formatted at 2026-08-09 02:16:16 on dist-test-slave-s11t"
server_key: "b3964c29242581d33be4f34fdff9da4c"
server_key_iv: "4dd5bb0c8836dcc1db1205f2494f0164"
server_key_version: "encryptionkey@0"
I20260809 02:16:17.954387 26323 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:16:17.966396 26323 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:16:17.969416 26323 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:16:17.969599 26323 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:16:17.970155 26323 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:16:17.970968 26323 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:16:17.971036 26323 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:17.971102 26323 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:16:17.971141 26323 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:17.980873 26323 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.130:36535
I20260809 02:16:17.980949 26526 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.130:36535 every 8 connection(s)
I20260809 02:16:17.981894 26323 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
I20260809 02:16:17.987728 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 26323
I20260809 02:16:17.987900 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 25331
Aug 09 02:16:18 dist-test-slave-s11t krb5kdc[24745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241777, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.75.130@KRBTEST.COM for oryx/127.15.75.189@KRBTEST.COM
I20260809 02:16:18.012305 25937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:17.985107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.130:52699 (local address 127.15.75.189:42709)
0809 02:16:17.985233 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:17.985238 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:17.985465 (+ 227us) server_negotiation.cc:407] Connection header received
0809 02:16:17.986504 (+ 1039us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:17.986506 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:17.986547 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:17.986648 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:18.001846 (+ 15198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:18.002574 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:18.004333 (+ 1759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:18.004677 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:18.004780 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:18.008137 (+ 3357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:18.008143 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:18.008149 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:18.008180 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:18.009876 (+ 1696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:18.010513 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:18.010519 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:18.010523 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:18.010600 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:18.010980 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:18.010986 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:18.010990 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:18.011224 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:18.011290 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:18.011940 (+ 650us) server_negotiation.cc:299] Negotiation successful
0809 02:16:18.012069 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 02:16:18.014037 26532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:17.984957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.189:42709 (local address 127.15.75.130:52699)
0809 02:16:17.985376 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:17.985390 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:17.986030 (+ 640us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:17.986772 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:17.986778 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:17.987152 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:17.987838 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:17.987851 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:18.002789 (+ 14938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:18.002795 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:18.004183 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:18.004199 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:18.005014 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:18.005020 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:18.005258 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:18.005948 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:18.005974 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:18.007937 (+ 1963us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:16:18.010024 (+ 2087us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:18.010033 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:18.010051 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:18.010407 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:18.010698 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:18.010704 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:18.010708 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:18.010870 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:18.011363 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:16:18.011371 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:16:18.011679 (+ 308us) client_negotiation.cc:769] Sending connection context
0809 02:16:18.012668 (+ 989us) client_negotiation.cc:240] Negotiation successful
0809 02:16:18.013041 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"mutex_wait_us":9,"thread_start_us":121,"threads_started":1}
I20260809 02:16:18.017859 26529 heartbeater.cc:347] Connected to a master server at 127.15.75.189:42709
I20260809 02:16:18.018178 26529 heartbeater.cc:464] Registering TS with master...
I20260809 02:16:18.018819 26529 heartbeater.cc:511] Master 127.15.75.189:42709 requested a full tablet report, sending...
I20260809 02:16:18.020044 25749 ts_manager.cc:194] Registered new tserver with Master: 6ee9ba5954274e238185dfb3cb465605 (127.15.75.130:36535)
Aug 09 02:16:18 dist-test-slave-s11t krb5kdc[24745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241777, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.75.130@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM
Aug 09 02:16:18 dist-test-slave-s11t krb5kdc[24745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241777, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.75.130@KRBTEST.COM for oryx/127.15.75.188@KRBTEST.COM
I20260809 02:16:18.024134 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.131:35531
--local_ip_for_outbound_sockets=127.15.75.131
--tserver_master_addrs=127.15.75.190:42501
--webserver_port=44281
--webserver_interface=127.15.75.131
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/krb5kdc/oryx_127.15.75.131.keytab
--principal=oryx/127.15.75.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:34317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.15.75.190:42501,127.15.75.189:42709
--tserver_master_addrs=127.15.75.190:42501,127.15.75.189:42709,127.15.75.188:40911 with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 02:16:18.027369 26531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:17.984393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:42501 (local address 127.15.75.130:37589)
0809 02:16:17.984908 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:17.984964 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:17.986030 (+ 1066us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:17.986592 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:17.986600 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:17.987170 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:17.987813 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:17.987833 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:18.018528 (+ 30695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:18.018534 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:18.019416 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:18.019427 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:18.020177 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:18.020181 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:18.020282 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:18.020863 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:18.020877 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:18.023091 (+ 2214us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:16:18.025378 (+ 2287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:18.025383 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:18.025385 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:18.025626 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:18.025993 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:18.025996 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:18.025998 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:18.026085 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:18.026549 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:16:18.026555 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:16:18.026643 (+ 88us) client_negotiation.cc:769] Sending connection context
0809 02:16:18.026739 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 02:16:18.026890 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"mutex_wait_us":180,"spinlock_wait_cycles":34176,"thread_start_us":111,"threads_started":1}
I20260809 02:16:18.027653 26321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:17.985504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.130:47959 (local address 127.15.75.188:40911)
0809 02:16:17.985628 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:17.985633 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:17.986592 (+ 959us) server_negotiation.cc:407] Connection header received
0809 02:16:17.986671 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:17.986674 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:17.986727 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:17.986809 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:17.988017 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:17.988857 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:18.018588 (+ 29731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:18.018963 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:18.019047 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:18.023280 (+ 4233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:18.023284 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:18.023292 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:18.023336 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:18.025472 (+ 2136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:18.025985 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:18.025990 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:18.025994 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:18.026093 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:18.026498 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:18.026501 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:18.026503 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:18.026725 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:18.026875 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:18.027268 (+ 393us) server_negotiation.cc:299] Negotiation successful
0809 02:16:18.027442 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 02:16:18.027660 26533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:17.985398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.188:40911 (local address 127.15.75.130:47959)
0809 02:16:17.986488 (+ 1090us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:17.986501 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:17.986593 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:17.986841 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:17.986844 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:17.987176 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:17.987870 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:17.987882 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:18.017466 (+ 29584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:18.017470 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:18.018418 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:18.018434 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:18.019201 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:18.019204 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:18.020997 (+ 1793us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:18.021485 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:18.021499 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:18.023091 (+ 1592us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:16:18.025587 (+ 2496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:18.025591 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:18.025593 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:18.025852 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:18.026214 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:18.026218 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:18.026221 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:18.026318 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:18.026953 (+ 635us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:16:18.026956 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:16:18.027038 (+ 82us) client_negotiation.cc:769] Sending connection context
0809 02:16:18.027145 (+ 107us) client_negotiation.cc:240] Negotiation successful
0809 02:16:18.027285 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1028,"mutex_wait_us":1791,"spinlock_wait_cycles":36992,"thread_start_us":87,"threads_started":1}
I20260809 02:16:18.028110 24883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:17.983663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.130:37589 (local address 127.15.75.190:42501)
0809 02:16:17.983829 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:17.983833 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:17.985069 (+ 1236us) server_negotiation.cc:407] Connection header received
0809 02:16:17.986264 (+ 1195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:17.986268 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:17.986313 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:17.986388 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:17.987988 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:17.988861 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:18.019595 (+ 30734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:18.019918 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:18.020023 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:18.023253 (+ 3230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:18.023259 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:18.023265 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:18.023301 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:18.025270 (+ 1969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:18.025745 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:18.025752 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:18.025756 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:18.025860 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:18.026186 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:18.026190 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:18.026191 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:18.026409 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:18.026501 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:18.027801 (+ 1300us) server_negotiation.cc:299] Negotiation successful
0809 02:16:18.027981 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 02:16:18.028343 26527 heartbeater.cc:347] Connected to a master server at 127.15.75.188:40911
I20260809 02:16:18.028438 26527 heartbeater.cc:464] Registering TS with master...
I20260809 02:16:18.028616 26527 heartbeater.cc:511] Master 127.15.75.188:40911 requested a full tablet report, sending...
I20260809 02:16:18.028767 26528 heartbeater.cc:347] Connected to a master server at 127.15.75.190:42501
I20260809 02:16:18.028867 26528 heartbeater.cc:464] Registering TS with master...
I20260809 02:16:18.029062 26528 heartbeater.cc:511] Master 127.15.75.190:42501 requested a full tablet report, sending...
I20260809 02:16:18.029824 26021 ts_manager.cc:194] Registered new tserver with Master: 6ee9ba5954274e238185dfb3cb465605 (127.15.75.130:36535)
I20260809 02:16:18.029810 24808 ts_manager.cc:194] Re-registered known tserver with Master: 6ee9ba5954274e238185dfb3cb465605 (127.15.75.130:36535)
I20260809 02:16:18.030488 24808 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.75.130@KRBTEST.COM'} at 127.15.75.130:37589
W20260809 02:16:18.141690 26535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:16:18.142076 26535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:16:18.142194 26535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:16:18.146597 26535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:16:18.146648 26535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:16:18.146744 26535 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.131
I20260809 02:16:18.152868 26535 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:34317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/krb5kdc/oryx_127.15.75.131.keytab
--principal=oryx/127.15.75.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.131:35531
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.75.131
--webserver_port=44281
--tserver_master_addrs=127.15.75.190:42501,127.15.75.189:42709,127.15.75.188:40911
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26535
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.131
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:16:18.154063 26535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:16:18.154467 26321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:18.150074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51416 (local address 127.15.75.188:40911)
0809 02:16:18.150262 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:18.150268 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:18.150361 (+ 93us) server_negotiation.cc:407] Connection header received
0809 02:16:18.150534 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:18.150537 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:18.150602 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:18.150697 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:16:18.150958 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:18.151775 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:18.153320 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:18.153945 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:18.154063 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:18.154223 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:18.154270 (+ 47us) server_negotiation.cc:299] Negotiation successful
0809 02:16:18.154344 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260809 02:16:18.154481 26542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:18.149978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.188:40911 (local address 127.0.0.1:51416)
0809 02:16:18.150265 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:18.150280 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:18.150409 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:18.150718 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:18.150721 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:18.150739 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:16:18.150854 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:18.150860 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:18.151897 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:18.151900 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:18.153203 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:18.153215 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:18.154072 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:18.154075 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:18.154162 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:18.154188 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 02:16:18.154257 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 02:16:18.154335 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":67,"threads_started":1}
I20260809 02:16:18.155143 26037 raft_consensus.cc:1203] T 00000000000000000000000000000000 P a39abb470ce448908175ec3b2e32e19f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260809 02:16:18.155320 26535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 02:16:18.155519 26084 sys_catalog.cc:455] T 00000000000000000000000000000000 P a39abb470ce448908175ec3b2e32e19f [sys.catalog]: SysCatalogTable state changed. Reason: New leader c59a0fdc35554648bd22e9b2280fe648. Latest consensus state: current_term: 1 leader_uuid: "c59a0fdc35554648bd22e9b2280fe648" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c59a0fdc35554648bd22e9b2280fe648" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 42501 } } peers { permanent_uuid: "b4f88c387e1d4b3c83ef38475e586f91" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 42709 } attrs { promote: false } } peers { permanent_uuid: "a39abb470ce448908175ec3b2e32e19f" member_type: NON_VOTER last_known_addr { host: "127.15.75.188" port: 40911 } attrs { promote: true } } }
I20260809 02:16:18.155644 26084 sys_catalog.cc:458] T 00000000000000000000000000000000 P a39abb470ce448908175ec3b2e32e19f [sys.catalog]: This master's current role is: LEARNER
W20260809 02:16:18.164654 26553 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:18.164677 26545 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:18.164726 26546 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:18.165066 26535 server_base.cc:1034] running on GCE node
I20260809 02:16:18.165467 26535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:16:18.166079 26535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:16:18.167255 26535 hybrid_clock.cc:648] HybridClock initialized: now 1786241778167243 us; error 37 us; skew 500 ppm
Aug 09 02:16:18 dist-test-slave-s11t krb5kdc[24745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241778, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.75.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:16:18.169965 26535 init.cc:377] Logged in from keytab as oryx/127.15.75.131@KRBTEST.COM (short username oryx)
I20260809 02:16:18.170742 26535 webserver.cc:460] Webserver started at http://127.15.75.131:44281/ using document root <none> and password file <none>
I20260809 02:16:18.171273 26535 fs_manager.cc:359] Metadata directory not provided
I20260809 02:16:18.171377 26535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:16:18.175478 26535 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260809 02:16:18.177655 26562 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:18.178805 26535 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 02:16:18.178949 26535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-2/wal
uuid: "acf85e824204477bb6d89218e1ed0533"
format_stamp: "Formatted at 2026-08-09 02:16:16 on dist-test-slave-s11t"
server_key: "d338c3d724e036a0dbb3c1de2790e409"
server_key_iv: "7a7dd79b575d687e3076948f2e7aa024"
server_key_version: "encryptionkey@0"
I20260809 02:16:18.179457 26535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:16:18.194967 26535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:16:18.197750 26535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:16:18.197950 26535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:16:18.198489 26535 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:16:18.199290 26535 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:16:18.199357 26535 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:18.199424 26535 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:16:18.199462 26535 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:18.208348 26535 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.131:35531
I20260809 02:16:18.208395 26732 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.131:35531 every 8 connection(s)
I20260809 02:16:18.209336 26535 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
I20260809 02:16:18.213995 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 26535
Aug 09 02:16:18 dist-test-slave-s11t krb5kdc[24745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241778, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.75.131@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM
Aug 09 02:16:18 dist-test-slave-s11t krb5kdc[24745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241778, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.75.131@KRBTEST.COM for oryx/127.15.75.188@KRBTEST.COM
Aug 09 02:16:18 dist-test-slave-s11t krb5kdc[24745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241778, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.75.131@KRBTEST.COM for oryx/127.15.75.189@KRBTEST.COM
I20260809 02:16:18.227638 26752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:18.217081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41184 (local address 127.15.75.189:42709)
0809 02:16:18.217871 (+ 790us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:18.217884 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:18.217899 (+ 15us) server_negotiation.cc:407] Connection header received
0809 02:16:18.218063 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:18.218067 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:18.218114 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:18.218176 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:18.219405 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:18.220173 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:18.221430 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:18.221742 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:18.221811 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:18.223627 (+ 1816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:18.223630 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:18.223633 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:18.223659 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:18.225541 (+ 1882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:18.226035 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:18.226038 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:18.226040 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:18.226139 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:18.226477 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:18.226484 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:18.226489 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:18.226705 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:18.226827 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:18.227176 (+ 349us) server_negotiation.cc:299] Negotiation successful
0809 02:16:18.227403 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":706,"thread_start_us":78,"threads_started":1}
I20260809 02:16:18.227988 26754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:18.217294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51424 (local address 127.15.75.188:40911)
0809 02:16:18.217588 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:18.217592 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:18.218203 (+ 611us) server_negotiation.cc:407] Connection header received
0809 02:16:18.218398 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:18.218404 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:18.218463 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:18.218531 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:18.219478 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:18.220187 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:18.221541 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:18.221943 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:18.222028 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:18.223870 (+ 1842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:18.223875 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:18.223881 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:18.223911 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:18.225809 (+ 1898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:18.226345 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:18.226362 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:18.226375 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:18.226481 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:18.226837 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:18.226842 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:18.226845 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:18.227084 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:18.227197 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:18.227613 (+ 416us) server_negotiation.cc:299] Negotiation successful
0809 02:16:18.227780 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":97,"threads_started":1}
I20260809 02:16:18.231539 24883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:18.211160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.131:60493 (local address 127.15.75.190:42501)
0809 02:16:18.211312 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:18.211319 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:18.212070 (+ 751us) server_negotiation.cc:407] Connection header received
0809 02:16:18.213287 (+ 1217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:18.213291 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:18.213363 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:18.213442 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:18.215907 (+ 2465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:18.216428 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:18.219382 (+ 2954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:18.219740 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:18.219840 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:18.225406 (+ 5566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:18.225410 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:18.225412 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:18.225442 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:18.228910 (+ 3468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:18.229576 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:18.229580 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:18.229581 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:18.229664 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:18.230103 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:18.230107 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:18.230112 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:18.230330 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:18.230445 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:18.231133 (+ 688us) server_negotiation.cc:299] Negotiation successful
0809 02:16:18.231305 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":17,"server-negotiator.queue_time_us":62}
I20260809 02:16:18.231835 26321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:18.212595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.131:55855 (local address 127.15.75.188:40911)
0809 02:16:18.212739 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:18.212744 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:18.212852 (+ 108us) server_negotiation.cc:407] Connection header received
0809 02:16:18.213304 (+ 452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:18.213307 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:18.213361 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:18.213417 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:18.215218 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:18.216047 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:18.220278 (+ 4231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:18.220602 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:18.220679 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:18.226234 (+ 5555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:18.226238 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:18.226240 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:18.226266 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:18.228530 (+ 2264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:18.230422 (+ 1892us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:18.230425 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:18.230427 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:18.230508 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:18.230849 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:18.230852 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:18.230854 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:18.231082 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:18.231173 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:18.231573 (+ 400us) server_negotiation.cc:299] Negotiation successful
0809 02:16:18.231701 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 02:16:18.232581 26740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:18.211436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:42501 (local address 127.15.75.131:60493)
0809 02:16:18.211916 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:18.211968 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:18.213083 (+ 1115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:18.213612 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:18.213619 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:18.214286 (+ 667us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:18.215063 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:18.215078 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:18.216545 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:18.216549 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:18.219220 (+ 2671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:18.219233 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:18.219998 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:18.220002 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:18.220203 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:18.220889 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:18.220909 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:18.225208 (+ 4299us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:16:18.229039 (+ 3831us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:18.229045 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:18.229063 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:18.229461 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:18.229783 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:18.229788 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:18.229791 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:18.229981 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:18.230470 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:16:18.230478 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:16:18.230834 (+ 356us) client_negotiation.cc:769] Sending connection context
0809 02:16:18.231088 (+ 254us) client_negotiation.cc:240] Negotiation successful
0809 02:16:18.231434 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"mutex_wait_us":179,"spinlock_wait_cycles":36096,"thread_start_us":145,"threads_started":1}
I20260809 02:16:18.233264 26741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:18.212445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.188:40911 (local address 127.15.75.131:55855)
0809 02:16:18.212762 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:18.212775 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:18.213082 (+ 307us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:18.213612 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:18.213621 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:18.214203 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:18.215047 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:18.215067 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:18.216232 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:18.216235 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:18.220127 (+ 3892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:18.220138 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:18.220867 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:18.220870 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:18.220995 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:18.222134 (+ 1139us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:18.222152 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:18.226067 (+ 3915us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:16:18.229805 (+ 3738us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:18.229807 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:18.229809 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:18.230302 (+ 493us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:18.230651 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:18.230654 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:18.230658 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:18.230753 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:18.231276 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:16:18.231279 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:16:18.231363 (+ 84us) client_negotiation.cc:769] Sending connection context
0809 02:16:18.232328 (+ 965us) client_negotiation.cc:240] Negotiation successful
0809 02:16:18.232467 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"mutex_wait_us":807,"spinlock_wait_cycles":14464,"thread_start_us":114,"threads_started":1}
I20260809 02:16:18.233983 26735 heartbeater.cc:347] Connected to a master server at 127.15.75.190:42501
I20260809 02:16:18.234242 26735 heartbeater.cc:464] Registering TS with master...
I20260809 02:16:18.234405 26733 heartbeater.cc:347] Connected to a master server at 127.15.75.188:40911
I20260809 02:16:18.234515 26733 heartbeater.cc:464] Registering TS with master...
I20260809 02:16:18.234839 26735 heartbeater.cc:511] Master 127.15.75.190:42501 requested a full tablet report, sending...
I20260809 02:16:18.234862 26733 heartbeater.cc:511] Master 127.15.75.188:40911 requested a full tablet report, sending...
I20260809 02:16:18.235318 26748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:18.216886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59572 (local address 127.15.75.190:42501)
0809 02:16:18.217471 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:18.217476 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:18.217732 (+ 256us) server_negotiation.cc:407] Connection header received
0809 02:16:18.217990 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:18.217995 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:18.218068 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:18.218145 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:18.219348 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:18.225724 (+ 6376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:18.227655 (+ 1931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:18.228232 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:18.228421 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:18.230250 (+ 1829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:18.230254 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:18.230256 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:18.230283 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:18.233033 (+ 2750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:18.233629 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:18.233632 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:18.233634 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:18.233724 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:18.234022 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:18.234025 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:18.234027 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:18.234351 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:18.234531 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:18.235044 (+ 513us) server_negotiation.cc:299] Negotiation successful
0809 02:16:18.235169 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":499,"thread_start_us":67,"threads_started":1}
I20260809 02:16:18.235466 25937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:18.211071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.131:42963 (local address 127.15.75.189:42709)
0809 02:16:18.211203 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:18.211208 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:18.212065 (+ 857us) server_negotiation.cc:407] Connection header received
0809 02:16:18.213303 (+ 1238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:18.213306 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:18.213361 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:18.213425 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:18.215218 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:18.216041 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:18.221225 (+ 5184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:18.221547 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:18.221615 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:18.230926 (+ 9311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:18.230929 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:18.230932 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:18.230959 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:18.233091 (+ 2132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:18.233767 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:18.233771 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:18.233773 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:18.233871 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:18.234429 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:18.234433 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:18.234436 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:18.234739 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:18.234836 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:18.235193 (+ 357us) server_negotiation.cc:299] Negotiation successful
0809 02:16:18.235318 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 02:16:18.235916 26739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:18.211435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.189:42709 (local address 127.15.75.131:42963)
0809 02:16:18.211916 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:18.211967 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:18.213082 (+ 1115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:18.213612 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:18.213621 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:18.214203 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:18.215049 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:18.215066 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:18.216232 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:18.216235 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:18.218293 (+ 2058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:18.218304 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:18.221789 (+ 3485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:18.221793 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:18.221902 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:18.222817 (+ 915us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:18.222838 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:18.229993 (+ 7155us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:16:18.233349 (+ 3356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:18.233354 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:18.233358 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:18.233663 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:18.234134 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:18.234137 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:18.234139 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:18.234306 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:18.234900 (+ 594us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:16:18.234903 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:16:18.234982 (+ 79us) client_negotiation.cc:769] Sending connection context
0809 02:16:18.235593 (+ 611us) client_negotiation.cc:240] Negotiation successful
0809 02:16:18.235745 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"mutex_wait_us":322,"spinlock_wait_cycles":21248,"thread_start_us":138,"threads_started":1}
I20260809 02:16:18.236022 24808 ts_manager.cc:194] Re-registered known tserver with Master: acf85e824204477bb6d89218e1ed0533 (127.15.75.131:35531)
I20260809 02:16:18.236114 26021 ts_manager.cc:194] Registered new tserver with Master: acf85e824204477bb6d89218e1ed0533 (127.15.75.131:35531)
I20260809 02:16:18.236487 26737 heartbeater.cc:347] Connected to a master server at 127.15.75.189:42709
I20260809 02:16:18.236562 26737 heartbeater.cc:464] Registering TS with master...
I20260809 02:16:18.236752 26737 heartbeater.cc:511] Master 127.15.75.189:42709 requested a full tablet report, sending...
I20260809 02:16:18.236792 24808 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.75.131@KRBTEST.COM'} at 127.15.75.131:60493
I20260809 02:16:18.237473 25749 ts_manager.cc:194] Registered new tserver with Master: acf85e824204477bb6d89218e1ed0533 (127.15.75.131:35531)
I20260809 02:16:18.239658 24808 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59572:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:16:18.242893 24808 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:16:18.245674 26760 sys_catalog.cc:455] T 00000000000000000000000000000000 P c59a0fdc35554648bd22e9b2280fe648 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c59a0fdc35554648bd22e9b2280fe648" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c59a0fdc35554648bd22e9b2280fe648" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 42501 } } peers { permanent_uuid: "b4f88c387e1d4b3c83ef38475e586f91" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 42709 } attrs { promote: false } } peers { permanent_uuid: "a39abb470ce448908175ec3b2e32e19f" member_type: NON_VOTER last_known_addr { host: "127.15.75.188" port: 40911 } attrs { promote: true } } }
I20260809 02:16:18.245816 26760 sys_catalog.cc:458] T 00000000000000000000000000000000 P c59a0fdc35554648bd22e9b2280fe648 [sys.catalog]: This master's current role is: LEADER
I20260809 02:16:18.250031 26761 raft_consensus.cc:1050] T 00000000000000000000000000000000 P c59a0fdc35554648bd22e9b2280fe648: attempting to promote NON_VOTER a39abb470ce448908175ec3b2e32e19f to VOTER
I20260809 02:16:18.250427 26761 consensus_queue.cc:237] T 00000000000000000000000000000000 P c59a0fdc35554648bd22e9b2280fe648 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c59a0fdc35554648bd22e9b2280fe648" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 42501 } } peers { permanent_uuid: "b4f88c387e1d4b3c83ef38475e586f91" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 42709 } attrs { promote: false } } peers { permanent_uuid: "a39abb470ce448908175ec3b2e32e19f" member_type: VOTER last_known_addr { host: "127.15.75.188" port: 40911 } attrs { promote: false } }
I20260809 02:16:18.251830 25766 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b4f88c387e1d4b3c83ef38475e586f91 [term 1 FOLLOWER]: Refusing update from remote peer c59a0fdc35554648bd22e9b2280fe648: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260809 02:16:18.252290 26047 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a39abb470ce448908175ec3b2e32e19f [term 1 LEARNER]: Refusing update from remote peer c59a0fdc35554648bd22e9b2280fe648: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260809 02:16:18.252629 26759 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c59a0fdc35554648bd22e9b2280fe648 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f88c387e1d4b3c83ef38475e586f91" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 42709 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260809 02:16:18.252820 26760 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c59a0fdc35554648bd22e9b2280fe648 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a39abb470ce448908175ec3b2e32e19f" member_type: VOTER last_known_addr { host: "127.15.75.188" port: 40911 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260809 02:16:18.254781 26759 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c59a0fdc35554648bd22e9b2280fe648 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, a39abb470ce448908175ec3b2e32e19f (127.15.75.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c59a0fdc35554648bd22e9b2280fe648" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 42501 } } peers { permanent_uuid: "b4f88c387e1d4b3c83ef38475e586f91" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 42709 } attrs { promote: false } } peers { permanent_uuid: "a39abb470ce448908175ec3b2e32e19f" member_type: VOTER last_known_addr { host: "127.15.75.188" port: 40911 } attrs { promote: false } } }
I20260809 02:16:18.255457 25766 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b4f88c387e1d4b3c83ef38475e586f91 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, a39abb470ce448908175ec3b2e32e19f (127.15.75.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c59a0fdc35554648bd22e9b2280fe648" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 42501 } } peers { permanent_uuid: "b4f88c387e1d4b3c83ef38475e586f91" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 42709 } attrs { promote: false } } peers { permanent_uuid: "a39abb470ce448908175ec3b2e32e19f" member_type: VOTER last_known_addr { host: "127.15.75.188" port: 40911 } attrs { promote: false } } }
I20260809 02:16:18.256562 26760 sys_catalog.cc:455] T 00000000000000000000000000000000 P c59a0fdc35554648bd22e9b2280fe648 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c59a0fdc35554648bd22e9b2280fe648" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c59a0fdc35554648bd22e9b2280fe648" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 42501 } } peers { permanent_uuid: "b4f88c387e1d4b3c83ef38475e586f91" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 42709 } attrs { promote: false } } peers { permanent_uuid: "a39abb470ce448908175ec3b2e32e19f" member_type: VOTER last_known_addr { host: "127.15.75.188" port: 40911 } attrs { promote: false } } }
I20260809 02:16:18.256654 26760 sys_catalog.cc:458] T 00000000000000000000000000000000 P c59a0fdc35554648bd22e9b2280fe648 [sys.catalog]: This master's current role is: LEADER
I20260809 02:16:18.256779 26760 sys_catalog.cc:455] T 00000000000000000000000000000000 P c59a0fdc35554648bd22e9b2280fe648 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c59a0fdc35554648bd22e9b2280fe648" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c59a0fdc35554648bd22e9b2280fe648" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 42501 } } peers { permanent_uuid: "b4f88c387e1d4b3c83ef38475e586f91" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 42709 } attrs { promote: false } } peers { permanent_uuid: "a39abb470ce448908175ec3b2e32e19f" member_type: VOTER last_known_addr { host: "127.15.75.188" port: 40911 } attrs { promote: false } } }
I20260809 02:16:18.256852 26760 sys_catalog.cc:458] T 00000000000000000000000000000000 P c59a0fdc35554648bd22e9b2280fe648 [sys.catalog]: This master's current role is: LEADER
I20260809 02:16:18.256490 26047 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a39abb470ce448908175ec3b2e32e19f [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, a39abb470ce448908175ec3b2e32e19f (127.15.75.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c59a0fdc35554648bd22e9b2280fe648" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 42501 } } peers { permanent_uuid: "b4f88c387e1d4b3c83ef38475e586f91" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 42709 } attrs { promote: false } } peers { permanent_uuid: "a39abb470ce448908175ec3b2e32e19f" member_type: VOTER last_known_addr { host: "127.15.75.188" port: 40911 } attrs { promote: false } } }
I20260809 02:16:18.257231 26761 sys_catalog.cc:455] T 00000000000000000000000000000000 P c59a0fdc35554648bd22e9b2280fe648 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c59a0fdc35554648bd22e9b2280fe648" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c59a0fdc35554648bd22e9b2280fe648" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 42501 } } peers { permanent_uuid: "b4f88c387e1d4b3c83ef38475e586f91" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 42709 } attrs { promote: false } } peers { permanent_uuid: "a39abb470ce448908175ec3b2e32e19f" member_type: VOTER last_known_addr { host: "127.15.75.188" port: 40911 } attrs { promote: false } } }
I20260809 02:16:18.257315 26761 sys_catalog.cc:458] T 00000000000000000000000000000000 P c59a0fdc35554648bd22e9b2280fe648 [sys.catalog]: This master's current role is: LEADER
I20260809 02:16:18.257678 26769 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4f88c387e1d4b3c83ef38475e586f91 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c59a0fdc35554648bd22e9b2280fe648" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c59a0fdc35554648bd22e9b2280fe648" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 42501 } } peers { permanent_uuid: "b4f88c387e1d4b3c83ef38475e586f91" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 42709 } attrs { promote: false } } peers { permanent_uuid: "a39abb470ce448908175ec3b2e32e19f" member_type: VOTER last_known_addr { host: "127.15.75.188" port: 40911 } attrs { promote: false } } }
I20260809 02:16:18.257828 26769 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4f88c387e1d4b3c83ef38475e586f91 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:16:18.258455 26084 sys_catalog.cc:455] T 00000000000000000000000000000000 P a39abb470ce448908175ec3b2e32e19f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c59a0fdc35554648bd22e9b2280fe648" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c59a0fdc35554648bd22e9b2280fe648" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 42501 } } peers { permanent_uuid: "b4f88c387e1d4b3c83ef38475e586f91" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 42709 } attrs { promote: false } } peers { permanent_uuid: "a39abb470ce448908175ec3b2e32e19f" member_type: VOTER last_known_addr { host: "127.15.75.188" port: 40911 } attrs { promote: false } } }
I20260809 02:16:18.258581 26084 sys_catalog.cc:458] T 00000000000000000000000000000000 P a39abb470ce448908175ec3b2e32e19f [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:16:18.267877 26542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:18.261400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.129:34105 (local address 127.0.0.1:51306)
0809 02:16:18.261720 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:18.261736 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:18.261822 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:18.262798 (+ 976us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:18.262801 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:18.262822 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:16:18.262948 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:18.262957 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:18.265092 (+ 2135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:18.265177 (+ 85us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:18.266338 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:18.266348 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:18.267458 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:18.267462 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:18.267546 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:18.267572 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 02:16:18.267634 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 02:16:18.267702 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"mutex_wait_us":33}
I20260809 02:16:18.267925 26772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:18.261742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51306 (local address 127.15.75.129:34105)
0809 02:16:18.262212 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:18.262230 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:18.262260 (+ 30us) server_negotiation.cc:407] Connection header received
0809 02:16:18.262347 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:18.262354 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:18.262607 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:18.263132 (+ 525us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:16:18.263861 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:18.264949 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:18.266468 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:18.267323 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:18.267458 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:18.267612 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:18.267668 (+ 56us) server_negotiation.cc:299] Negotiation successful
0809 02:16:18.267755 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":104,"threads_started":1}
I20260809 02:16:18.268427 26770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:18.261690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.131:35531 (local address 127.0.0.1:35478)
0809 02:16:18.261966 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:18.261982 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:18.262078 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:18.263294 (+ 1216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:18.263297 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:18.263316 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:16:18.263533 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:18.263541 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:18.265014 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:18.265018 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:18.266493 (+ 1475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:18.266504 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:18.267847 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:18.267850 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:18.268152 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:18.268189 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 02:16:18.268261 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 02:16:18.268334 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"mutex_wait_us":33,"thread_start_us":130,"threads_started":1}
I20260809 02:16:18.269059 26776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:18.262277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35478 (local address 127.15.75.131:35531)
0809 02:16:18.262851 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:18.262864 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:18.262897 (+ 33us) server_negotiation.cc:407] Connection header received
0809 02:16:18.262971 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:18.262976 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:18.263160 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:18.263316 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:16:18.263646 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:18.264892 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:18.266622 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:18.267385 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:18.267476 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:18.267612 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:18.268793 (+ 1181us) server_negotiation.cc:299] Negotiation successful
0809 02:16:18.268915 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":470,"thread_start_us":135,"threads_started":1}
I20260809 02:16:18.269658 26775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:18.262049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.130:36535 (local address 127.0.0.1:36998)
0809 02:16:18.262479 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:18.262492 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:18.262576 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:18.265298 (+ 2722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:18.265301 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:18.265320 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:16:18.265405 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:18.265414 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:18.266873 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:18.266878 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:18.267713 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:18.267721 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:18.269267 (+ 1546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:18.269270 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:18.269373 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:18.269400 (+ 27us) client_negotiation.cc:769] Sending connection context
0809 02:16:18.269453 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 02:16:18.269527 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":86,"threads_started":1}
I20260809 02:16:18.270219 26777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:18.262460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36998 (local address 127.15.75.130:36535)
0809 02:16:18.264828 (+ 2368us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:18.264844 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:18.264867 (+ 23us) server_negotiation.cc:407] Connection header received
0809 02:16:18.264936 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:18.264939 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:18.265132 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:18.265576 (+ 444us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:16:18.265732 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:18.266757 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:18.267844 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:18.268637 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:18.269863 (+ 1226us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:18.269992 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:18.270033 (+ 41us) server_negotiation.cc:299] Negotiation successful
0809 02:16:18.270110 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1043,"thread_start_us":107,"threads_started":1}
I20260809 02:16:18.271407 26206 tablet_service.cc:1467] Processing CreateTablet for tablet 7a51e3d948e74141b52f1b8ff7f8e751 (DEFAULT_TABLE table=test-table [id=cea7bc56b0124c9390b38d014acc8f28]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:16:18.271656 26633 tablet_service.cc:1467] Processing CreateTablet for tablet 7a51e3d948e74141b52f1b8ff7f8e751 (DEFAULT_TABLE table=test-table [id=cea7bc56b0124c9390b38d014acc8f28]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:16:18.272606 26633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a51e3d948e74141b52f1b8ff7f8e751. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:16:18.272346 26424 tablet_service.cc:1467] Processing CreateTablet for tablet 7a51e3d948e74141b52f1b8ff7f8e751 (DEFAULT_TABLE table=test-table [id=cea7bc56b0124c9390b38d014acc8f28]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:16:18.273046 26206 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a51e3d948e74141b52f1b8ff7f8e751. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:16:18.273407 26424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a51e3d948e74141b52f1b8ff7f8e751. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:16:18.280422 26778 tablet_bootstrap.cc:492] T 7a51e3d948e74141b52f1b8ff7f8e751 P 6ee9ba5954274e238185dfb3cb465605: Bootstrap starting.
I20260809 02:16:18.282621 26778 tablet_bootstrap.cc:654] T 7a51e3d948e74141b52f1b8ff7f8e751 P 6ee9ba5954274e238185dfb3cb465605: Neither blocks nor log segments found. Creating new log.
I20260809 02:16:18.283541 26778 log.cc:824] T 7a51e3d948e74141b52f1b8ff7f8e751 P 6ee9ba5954274e238185dfb3cb465605: Log is configured to *not* fsync() on all Append() calls
I20260809 02:16:18.284214 26779 tablet_bootstrap.cc:492] T 7a51e3d948e74141b52f1b8ff7f8e751 P acf85e824204477bb6d89218e1ed0533: Bootstrap starting.
I20260809 02:16:18.285488 26778 tablet_bootstrap.cc:492] T 7a51e3d948e74141b52f1b8ff7f8e751 P 6ee9ba5954274e238185dfb3cb465605: No bootstrap required, opened a new log
I20260809 02:16:18.285701 26778 ts_tablet_manager.cc:1397] T 7a51e3d948e74141b52f1b8ff7f8e751 P 6ee9ba5954274e238185dfb3cb465605: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 02:16:18.285974 26779 tablet_bootstrap.cc:654] T 7a51e3d948e74141b52f1b8ff7f8e751 P acf85e824204477bb6d89218e1ed0533: Neither blocks nor log segments found. Creating new log.
I20260809 02:16:18.286602 26779 log.cc:824] T 7a51e3d948e74141b52f1b8ff7f8e751 P acf85e824204477bb6d89218e1ed0533: Log is configured to *not* fsync() on all Append() calls
I20260809 02:16:18.287242 26780 tablet_bootstrap.cc:492] T 7a51e3d948e74141b52f1b8ff7f8e751 P 6babbe94e6d54368a5c7edf6c5847ba1: Bootstrap starting.
I20260809 02:16:18.288625 26779 tablet_bootstrap.cc:492] T 7a51e3d948e74141b52f1b8ff7f8e751 P acf85e824204477bb6d89218e1ed0533: No bootstrap required, opened a new log
I20260809 02:16:18.288836 26780 tablet_bootstrap.cc:654] T 7a51e3d948e74141b52f1b8ff7f8e751 P 6babbe94e6d54368a5c7edf6c5847ba1: Neither blocks nor log segments found. Creating new log.
I20260809 02:16:18.288892 26779 ts_tablet_manager.cc:1397] T 7a51e3d948e74141b52f1b8ff7f8e751 P acf85e824204477bb6d89218e1ed0533: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.002s
I20260809 02:16:18.289498 26780 log.cc:824] T 7a51e3d948e74141b52f1b8ff7f8e751 P 6babbe94e6d54368a5c7edf6c5847ba1: Log is configured to *not* fsync() on all Append() calls
I20260809 02:16:18.290520 26778 raft_consensus.cc:348] T 7a51e3d948e74141b52f1b8ff7f8e751 P 6ee9ba5954274e238185dfb3cb465605 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf85e824204477bb6d89218e1ed0533" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 35531 } } peers { permanent_uuid: "6babbe94e6d54368a5c7edf6c5847ba1" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 34105 } } peers { permanent_uuid: "6ee9ba5954274e238185dfb3cb465605" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 36535 } }
I20260809 02:16:18.290803 26778 raft_consensus.cc:374] T 7a51e3d948e74141b52f1b8ff7f8e751 P 6ee9ba5954274e238185dfb3cb465605 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:16:18.290874 26778 raft_consensus.cc:729] T 7a51e3d948e74141b52f1b8ff7f8e751 P 6ee9ba5954274e238185dfb3cb465605 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ee9ba5954274e238185dfb3cb465605, State: Initialized, Role: FOLLOWER
I20260809 02:16:18.291615 26780 tablet_bootstrap.cc:492] T 7a51e3d948e74141b52f1b8ff7f8e751 P 6babbe94e6d54368a5c7edf6c5847ba1: No bootstrap required, opened a new log
I20260809 02:16:18.291544 26778 consensus_queue.cc:260] T 7a51e3d948e74141b52f1b8ff7f8e751 P 6ee9ba5954274e238185dfb3cb465605 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf85e824204477bb6d89218e1ed0533" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 35531 } } peers { permanent_uuid: "6babbe94e6d54368a5c7edf6c5847ba1" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 34105 } } peers { permanent_uuid: "6ee9ba5954274e238185dfb3cb465605" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 36535 } }
I20260809 02:16:18.291800 26780 ts_tablet_manager.cc:1397] T 7a51e3d948e74141b52f1b8ff7f8e751 P 6babbe94e6d54368a5c7edf6c5847ba1: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260809 02:16:18.292723 26778 ts_tablet_manager.cc:1428] T 7a51e3d948e74141b52f1b8ff7f8e751 P 6ee9ba5954274e238185dfb3cb465605: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260809 02:16:18.293929 26528 heartbeater.cc:503] Master 127.15.75.190:42501 was elected leader, sending a full tablet report...
I20260809 02:16:18.296918 26779 raft_consensus.cc:348] T 7a51e3d948e74141b52f1b8ff7f8e751 P acf85e824204477bb6d89218e1ed0533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf85e824204477bb6d89218e1ed0533" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 35531 } } peers { permanent_uuid: "6babbe94e6d54368a5c7edf6c5847ba1" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 34105 } } peers { permanent_uuid: "6ee9ba5954274e238185dfb3cb465605" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 36535 } }
I20260809 02:16:18.297168 26779 raft_consensus.cc:374] T 7a51e3d948e74141b52f1b8ff7f8e751 P acf85e824204477bb6d89218e1ed0533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:16:18.297236 26779 raft_consensus.cc:729] T 7a51e3d948e74141b52f1b8ff7f8e751 P acf85e824204477bb6d89218e1ed0533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acf85e824204477bb6d89218e1ed0533, State: Initialized, Role: FOLLOWER
I20260809 02:16:18.297323 26780 raft_consensus.cc:348] T 7a51e3d948e74141b52f1b8ff7f8e751 P 6babbe94e6d54368a5c7edf6c5847ba1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf85e824204477bb6d89218e1ed0533" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 35531 } } peers { permanent_uuid: "6babbe94e6d54368a5c7edf6c5847ba1" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 34105 } } peers { permanent_uuid: "6ee9ba5954274e238185dfb3cb465605" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 36535 } }
I20260809 02:16:18.297524 26780 raft_consensus.cc:374] T 7a51e3d948e74141b52f1b8ff7f8e751 P 6babbe94e6d54368a5c7edf6c5847ba1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:16:18.297609 26780 raft_consensus.cc:729] T 7a51e3d948e74141b52f1b8ff7f8e751 P 6babbe94e6d54368a5c7edf6c5847ba1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6babbe94e6d54368a5c7edf6c5847ba1, State: Initialized, Role: FOLLOWER
I20260809 02:16:18.297672 26779 consensus_queue.cc:260] T 7a51e3d948e74141b52f1b8ff7f8e751 P acf85e824204477bb6d89218e1ed0533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf85e824204477bb6d89218e1ed0533" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 35531 } } peers { permanent_uuid: "6babbe94e6d54368a5c7edf6c5847ba1" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 34105 } } peers { permanent_uuid: "6ee9ba5954274e238185dfb3cb465605" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 36535 } }
I20260809 02:16:18.298003 26780 consensus_queue.cc:260] T 7a51e3d948e74141b52f1b8ff7f8e751 P 6babbe94e6d54368a5c7edf6c5847ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf85e824204477bb6d89218e1ed0533" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 35531 } } peers { permanent_uuid: "6babbe94e6d54368a5c7edf6c5847ba1" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 34105 } } peers { permanent_uuid: "6ee9ba5954274e238185dfb3cb465605" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 36535 } }
I20260809 02:16:18.298336 26735 heartbeater.cc:503] Master 127.15.75.190:42501 was elected leader, sending a full tablet report...
I20260809 02:16:18.298658 26779 ts_tablet_manager.cc:1428] T 7a51e3d948e74141b52f1b8ff7f8e751 P acf85e824204477bb6d89218e1ed0533: Time spent starting tablet: real 0.010s user 0.004s sys 0.002s
I20260809 02:16:18.298919 26780 ts_tablet_manager.cc:1428] T 7a51e3d948e74141b52f1b8ff7f8e751 P 6babbe94e6d54368a5c7edf6c5847ba1: Time spent starting tablet: real 0.007s user 0.000s sys 0.006s
I20260809 02:16:18.301154 26312 heartbeater.cc:503] Master 127.15.75.190:42501 was elected leader, sending a full tablet report...
W20260809 02:16:18.460599 26738 tablet.cc:2369] T 7a51e3d948e74141b52f1b8ff7f8e751 P acf85e824204477bb6d89218e1ed0533: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 02:16:18.483556 26530 tablet.cc:2369] T 7a51e3d948e74141b52f1b8ff7f8e751 P 6ee9ba5954274e238185dfb3cb465605: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 02:16:18.521724 26315 tablet.cc:2369] T 7a51e3d948e74141b52f1b8ff7f8e751 P 6babbe94e6d54368a5c7edf6c5847ba1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:16:18.646588 26786 raft_consensus.cc:482] T 7a51e3d948e74141b52f1b8ff7f8e751 P 6ee9ba5954274e238185dfb3cb465605 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:16:18.646879 26786 raft_consensus.cc:504] T 7a51e3d948e74141b52f1b8ff7f8e751 P 6ee9ba5954274e238185dfb3cb465605 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf85e824204477bb6d89218e1ed0533" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 35531 } } peers { permanent_uuid: "6babbe94e6d54368a5c7edf6c5847ba1" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 34105 } } peers { permanent_uuid: "6ee9ba5954274e238185dfb3cb465605" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 36535 } }
I20260809 02:16:18.648092 26786 leader_election.cc:288] T 7a51e3d948e74141b52f1b8ff7f8e751 P 6ee9ba5954274e238185dfb3cb465605 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acf85e824204477bb6d89218e1ed0533 (127.15.75.131:35531), 6babbe94e6d54368a5c7edf6c5847ba1 (127.15.75.129:34105)
I20260809 02:16:18.652243 26776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:18.648444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.130:35347 (local address 127.15.75.131:35531)
0809 02:16:18.648590 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:18.648596 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:18.648715 (+ 119us) server_negotiation.cc:407] Connection header received
0809 02:16:18.648887 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:18.648890 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:18.648943 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:18.649011 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:16:18.649326 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:18.649897 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:18.651185 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:18.651565 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:18.651659 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:18.651699 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:18.652031 (+ 332us) server_negotiation.cc:299] Negotiation successful
0809 02:16:18.652109 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 02:16:18.652279 26791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:18.648419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.129:34105 (local address 127.15.75.130:53967)
0809 02:16:18.648668 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:18.648701 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:18.648763 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:18.649196 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:18.649198 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:18.649215 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:16:18.649343 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:18.649350 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:18.650283 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:18.650286 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:18.651023 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:18.651031 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:18.651798 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:18.651806 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:18.651944 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:18.651983 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 02:16:18.652064 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 02:16:18.652128 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"mutex_wait_us":42,"thread_start_us":67,"threads_started":1}
I20260809 02:16:18.652323 26772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:18.648796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.130:53967 (local address 127.15.75.129:34105)
0809 02:16:18.648946 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:18.648949 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:18.648961 (+ 12us) server_negotiation.cc:407] Connection header received
0809 02:16:18.649031 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:18.649035 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:18.649083 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:18.649162 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:16:18.649489 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:18.650138 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:18.651152 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:18.651643 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:18.651776 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:18.651842 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:18.652101 (+ 259us) server_negotiation.cc:299] Negotiation successful
0809 02:16:18.652201 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 02:16:18.652287 26790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:18.648299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.131:35531 (local address 127.15.75.130:35347)
0809 02:16:18.648589 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:18.648604 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:18.648761 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:18.649055 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:18.649058 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:18.649076 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:16:18.649202 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:18.649209 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:18.650039 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:18.650042 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:18.651020 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:18.651030 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:18.651714 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:18.651721 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:18.651898 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:18.651919 (+ 21us) client_negotiation.cc:769] Sending connection context
0809 02:16:18.651985 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 02:16:18.652078 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"mutex_wait_us":29,"thread_start_us":104,"threads_started":1}
I20260809 02:16:18.653041 26233 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a51e3d948e74141b52f1b8ff7f8e751" candidate_uuid: "6ee9ba5954274e238185dfb3cb465605" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6babbe94e6d54368a5c7edf6c5847ba1" is_pre_election: true
I20260809 02:16:18.653249 26665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a51e3d948e74141b52f1b8ff7f8e751" candidate_uuid: "6ee9ba5954274e238185dfb3cb465605" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf85e824204477bb6d89218e1ed0533" is_pre_election: true
I20260809 02:16:18.653362 26233 raft_consensus.cc:2454] T 7a51e3d948e74141b52f1b8ff7f8e751 P 6babbe94e6d54368a5c7edf6c5847ba1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ee9ba5954274e238185dfb3cb465605 in term 0.
I20260809 02:16:18.653504 26665 raft_consensus.cc:2454] T 7a51e3d948e74141b52f1b8ff7f8e751 P acf85e824204477bb6d89218e1ed0533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ee9ba5954274e238185dfb3cb465605 in term 0.
I20260809 02:16:18.653978 26342 leader_election.cc:302] T 7a51e3d948e74141b52f1b8ff7f8e751 P 6ee9ba5954274e238185dfb3cb465605 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ee9ba5954274e238185dfb3cb465605, acf85e824204477bb6d89218e1ed0533; no voters:
I20260809 02:16:18.654398 26786 raft_consensus.cc:2781] T 7a51e3d948e74141b52f1b8ff7f8e751 P 6ee9ba5954274e238185dfb3cb465605 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:16:18.654469 26786 raft_consensus.cc:482] T 7a51e3d948e74141b52f1b8ff7f8e751 P 6ee9ba5954274e238185dfb3cb465605 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:16:18.654498 26786 raft_consensus.cc:3037] T 7a51e3d948e74141b52f1b8ff7f8e751 P 6ee9ba5954274e238185dfb3cb465605 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:16:18.655473 26786 raft_consensus.cc:504] T 7a51e3d948e74141b52f1b8ff7f8e751 P 6ee9ba5954274e238185dfb3cb465605 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf85e824204477bb6d89218e1ed0533" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 35531 } } peers { permanent_uuid: "6babbe94e6d54368a5c7edf6c5847ba1" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 34105 } } peers { permanent_uuid: "6ee9ba5954274e238185dfb3cb465605" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 36535 } }
I20260809 02:16:18.655788 26786 leader_election.cc:288] T 7a51e3d948e74141b52f1b8ff7f8e751 P 6ee9ba5954274e238185dfb3cb465605 [CANDIDATE]: Term 1 election: Requested vote from peers acf85e824204477bb6d89218e1ed0533 (127.15.75.131:35531), 6babbe94e6d54368a5c7edf6c5847ba1 (127.15.75.129:34105)
I20260809 02:16:18.656142 26233 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a51e3d948e74141b52f1b8ff7f8e751" candidate_uuid: "6ee9ba5954274e238185dfb3cb465605" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6babbe94e6d54368a5c7edf6c5847ba1"
I20260809 02:16:18.656160 26665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a51e3d948e74141b52f1b8ff7f8e751" candidate_uuid: "6ee9ba5954274e238185dfb3cb465605" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf85e824204477bb6d89218e1ed0533"
I20260809 02:16:18.656275 26233 raft_consensus.cc:3037] T 7a51e3d948e74141b52f1b8ff7f8e751 P 6babbe94e6d54368a5c7edf6c5847ba1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:16:18.656291 26665 raft_consensus.cc:3037] T 7a51e3d948e74141b52f1b8ff7f8e751 P acf85e824204477bb6d89218e1ed0533 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:16:18.657570 26233 raft_consensus.cc:2454] T 7a51e3d948e74141b52f1b8ff7f8e751 P 6babbe94e6d54368a5c7edf6c5847ba1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ee9ba5954274e238185dfb3cb465605 in term 1.
I20260809 02:16:18.657590 26665 raft_consensus.cc:2454] T 7a51e3d948e74141b52f1b8ff7f8e751 P acf85e824204477bb6d89218e1ed0533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ee9ba5954274e238185dfb3cb465605 in term 1.
I20260809 02:16:18.657876 26342 leader_election.cc:302] T 7a51e3d948e74141b52f1b8ff7f8e751 P 6ee9ba5954274e238185dfb3cb465605 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ee9ba5954274e238185dfb3cb465605, acf85e824204477bb6d89218e1ed0533; no voters:
I20260809 02:16:18.658116 26786 raft_consensus.cc:2781] T 7a51e3d948e74141b52f1b8ff7f8e751 P 6ee9ba5954274e238185dfb3cb465605 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:16:18.658334 26786 raft_consensus.cc:686] T 7a51e3d948e74141b52f1b8ff7f8e751 P 6ee9ba5954274e238185dfb3cb465605 [term 1 LEADER]: Becoming Leader. State: Replica: 6ee9ba5954274e238185dfb3cb465605, State: Running, Role: LEADER
I20260809 02:16:18.658666 26786 consensus_queue.cc:237] T 7a51e3d948e74141b52f1b8ff7f8e751 P 6ee9ba5954274e238185dfb3cb465605 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf85e824204477bb6d89218e1ed0533" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 35531 } } peers { permanent_uuid: "6babbe94e6d54368a5c7edf6c5847ba1" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 34105 } } peers { permanent_uuid: "6ee9ba5954274e238185dfb3cb465605" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 36535 } }
I20260809 02:16:18.662120 24808 catalog_manager.cc:5495] T 7a51e3d948e74141b52f1b8ff7f8e751 P 6ee9ba5954274e238185dfb3cb465605 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6ee9ba5954274e238185dfb3cb465605 (127.15.75.130). New cstate: current_term: 1 leader_uuid: "6ee9ba5954274e238185dfb3cb465605" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf85e824204477bb6d89218e1ed0533" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 35531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6babbe94e6d54368a5c7edf6c5847ba1" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 34105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ee9ba5954274e238185dfb3cb465605" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 36535 } health_report { overall_health: HEALTHY } } }
I20260809 02:16:18.718551 26777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:18.714534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37004 (local address 127.15.75.130:36535)
0809 02:16:18.714685 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:18.714690 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:18.714707 (+ 17us) server_negotiation.cc:407] Connection header received
0809 02:16:18.714911 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:18.714914 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:18.714974 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:18.715101 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:16:18.715393 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:18.715962 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:18.716982 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:18.717224 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:18.717340 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:18.717683 (+ 343us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:16:18.718122 (+ 439us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:16:18.718267 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:18.718338 (+ 71us) server_negotiation.cc:299] Negotiation successful
0809 02:16:18.718398 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 02:16:18.724608 26233 raft_consensus.cc:1261] T 7a51e3d948e74141b52f1b8ff7f8e751 P 6babbe94e6d54368a5c7edf6c5847ba1 [term 1 FOLLOWER]: Refusing update from remote peer 6ee9ba5954274e238185dfb3cb465605: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:16:18.724608 26665 raft_consensus.cc:1261] T 7a51e3d948e74141b52f1b8ff7f8e751 P acf85e824204477bb6d89218e1ed0533 [term 1 FOLLOWER]: Refusing update from remote peer 6ee9ba5954274e238185dfb3cb465605: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:16:18.725368 26786 consensus_queue.cc:1035] T 7a51e3d948e74141b52f1b8ff7f8e751 P 6ee9ba5954274e238185dfb3cb465605 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf85e824204477bb6d89218e1ed0533" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 35531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:16:18.725620 26792 consensus_queue.cc:1035] T 7a51e3d948e74141b52f1b8ff7f8e751 P 6ee9ba5954274e238185dfb3cb465605 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6babbe94e6d54368a5c7edf6c5847ba1" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 34105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:16:18.733423 26795 mvcc.cc:205] Tried to move back new op lower bound from 7316446325648117760 to 7316446325388619776. Current Snapshot: MvccSnapshot[applied={T|T < 7316446325648117760}]
I20260809 02:16:18.735571 26798 mvcc.cc:205] Tried to move back new op lower bound from 7316446325648117760 to 7316446325388619776. Current Snapshot: MvccSnapshot[applied={T|T < 7316446325648117760}]
I20260809 02:16:18.739727 26797 mvcc.cc:205] Tried to move back new op lower bound from 7316446325648117760 to 7316446325388619776. Current Snapshot: MvccSnapshot[applied={T|T < 7316446325648117760 or (T in {7316446325648117760})}]
I20260809 02:16:18.746387 24808 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59572:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:16:18.746556 24808 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59572:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:16:18.751072 24808 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c59a0fdc35554648bd22e9b2280fe648: Sending DeleteTablet for 3 replicas of tablet 7a51e3d948e74141b52f1b8ff7f8e751
I20260809 02:16:18.751973 26633 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a51e3d948e74141b52f1b8ff7f8e751 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:16:18 UTC) from {username='oryx', principal='oryx/127.15.75.190@KRBTEST.COM'} at 127.0.0.1:35478
I20260809 02:16:18.752032 26424 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a51e3d948e74141b52f1b8ff7f8e751 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:16:18 UTC) from {username='oryx', principal='oryx/127.15.75.190@KRBTEST.COM'} at 127.0.0.1:36998
I20260809 02:16:18.752413 26813 tablet_replica.cc:330] T 7a51e3d948e74141b52f1b8ff7f8e751 P acf85e824204477bb6d89218e1ed0533: stopping tablet replica
I20260809 02:16:18.752426 26206 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a51e3d948e74141b52f1b8ff7f8e751 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:16:18 UTC) from {username='oryx', principal='oryx/127.15.75.190@KRBTEST.COM'} at 127.0.0.1:51306
I20260809 02:16:18.752516 26814 tablet_replica.cc:330] T 7a51e3d948e74141b52f1b8ff7f8e751 P 6ee9ba5954274e238185dfb3cb465605: stopping tablet replica
I20260809 02:16:18.752880 26813 raft_consensus.cc:2229] T 7a51e3d948e74141b52f1b8ff7f8e751 P acf85e824204477bb6d89218e1ed0533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:16:18.752880 26814 raft_consensus.cc:2229] T 7a51e3d948e74141b52f1b8ff7f8e751 P 6ee9ba5954274e238185dfb3cb465605 [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:16:18.753269 26813 raft_consensus.cc:2258] T 7a51e3d948e74141b52f1b8ff7f8e751 P acf85e824204477bb6d89218e1ed0533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:16:18.753283 26814 raft_consensus.cc:2258] T 7a51e3d948e74141b52f1b8ff7f8e751 P 6ee9ba5954274e238185dfb3cb465605 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:16:18.754997 26813 ts_tablet_manager.cc:1905] T 7a51e3d948e74141b52f1b8ff7f8e751 P acf85e824204477bb6d89218e1ed0533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:16:18.756115 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 24763
I20260809 02:16:18.761251 26814 ts_tablet_manager.cc:1905] T 7a51e3d948e74141b52f1b8ff7f8e751 P 6ee9ba5954274e238185dfb3cb465605: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:16:18.762598 26813 ts_tablet_manager.cc:1918] T 7a51e3d948e74141b52f1b8ff7f8e751 P acf85e824204477bb6d89218e1ed0533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:16:18.762799 26813 log.cc:1196] T 7a51e3d948e74141b52f1b8ff7f8e751 P acf85e824204477bb6d89218e1ed0533: Deleting WAL directory at /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-2/wal/wals/7a51e3d948e74141b52f1b8ff7f8e751
I20260809 02:16:18.763214 26813 ts_tablet_manager.cc:1939] T 7a51e3d948e74141b52f1b8ff7f8e751 P acf85e824204477bb6d89218e1ed0533: Deleting consensus metadata
I20260809 02:16:18.764015 26814 ts_tablet_manager.cc:1918] T 7a51e3d948e74141b52f1b8ff7f8e751 P 6ee9ba5954274e238185dfb3cb465605: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:16:18.764122 26814 log.cc:1196] T 7a51e3d948e74141b52f1b8ff7f8e751 P 6ee9ba5954274e238185dfb3cb465605: Deleting WAL directory at /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-1/wal/wals/7a51e3d948e74141b52f1b8ff7f8e751
I20260809 02:16:18.764458 26814 ts_tablet_manager.cc:1939] T 7a51e3d948e74141b52f1b8ff7f8e751 P 6ee9ba5954274e238185dfb3cb465605: Deleting consensus metadata
I20260809 02:16:18.765545 26815 tablet_replica.cc:330] T 7a51e3d948e74141b52f1b8ff7f8e751 P 6babbe94e6d54368a5c7edf6c5847ba1: stopping tablet replica
I20260809 02:16:18.765858 26815 raft_consensus.cc:2229] T 7a51e3d948e74141b52f1b8ff7f8e751 P 6babbe94e6d54368a5c7edf6c5847ba1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:16:18.766175 26815 raft_consensus.cc:2258] T 7a51e3d948e74141b52f1b8ff7f8e751 P 6babbe94e6d54368a5c7edf6c5847ba1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:16:18.767417 26815 ts_tablet_manager.cc:1905] T 7a51e3d948e74141b52f1b8ff7f8e751 P 6babbe94e6d54368a5c7edf6c5847ba1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:16:18.769939 26815 ts_tablet_manager.cc:1918] T 7a51e3d948e74141b52f1b8ff7f8e751 P 6babbe94e6d54368a5c7edf6c5847ba1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:16:18.770032 26815 log.cc:1196] T 7a51e3d948e74141b52f1b8ff7f8e751 P 6babbe94e6d54368a5c7edf6c5847ba1: Deleting WAL directory at /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-0/wal/wals/7a51e3d948e74141b52f1b8ff7f8e751
I20260809 02:16:18.770345 26815 ts_tablet_manager.cc:1939] T 7a51e3d948e74141b52f1b8ff7f8e751 P 6babbe94e6d54368a5c7edf6c5847ba1: Deleting consensus metadata
W20260809 02:16:18.778172 26342 connection.cc:535] server connection from 127.0.0.1:36998 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 02:16:18.778188 26567 connection.cc:535] server connection from 127.0.0.1:35478 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 02:16:18.778218 26132 connection.cc:535] server connection from 127.0.0.1:51306 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 02:16:18.778604 25734 connection.cc:535] server connection from 127.0.0.1:41178 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 02:16:18.786165 26817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:18.754437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41200 (local address 127.15.75.189:42709)
0809 02:16:18.768766 (+ 14329us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:18.768774 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:18.768798 (+ 24us) server_negotiation.cc:407] Connection header received
0809 02:16:18.768856 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:18.768863 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:18.768931 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:18.769045 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:18.773746 (+ 4701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:18.774576 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:18.775853 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:18.776208 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:18.776294 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:18.782020 (+ 5726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:18.782024 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:18.782027 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:18.782053 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:18.783859 (+ 1806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:18.784356 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:18.784359 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:18.784361 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:18.784439 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:18.784747 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:18.784750 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:18.784752 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:18.784957 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:18.785768 (+ 811us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:18.785851 (+ 83us) server_negotiation.cc:299] Negotiation successful
0809 02:16:18.785981 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14195,"thread_start_us":65,"threads_started":1}
I20260809 02:16:18.795992 26823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:18.782237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41216 (local address 127.15.75.189:42709)
0809 02:16:18.783241 (+ 1004us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:18.783246 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:18.783260 (+ 14us) server_negotiation.cc:407] Connection header received
0809 02:16:18.783297 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:18.783300 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:18.783346 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:18.783431 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:18.784372 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:18.785298 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:18.789776 (+ 4478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:18.790001 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:18.790107 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:18.791630 (+ 1523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:18.791634 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:18.791637 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:18.791667 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:18.793730 (+ 2063us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:18.794235 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:18.794240 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:18.794244 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:18.794347 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:18.795024 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:18.795028 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:18.795029 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:18.795242 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:18.795366 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:18.795674 (+ 308us) server_negotiation.cc:299] Negotiation successful
0809 02:16:18.795820 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":530,"thread_start_us":81,"threads_started":1}
I20260809 02:16:18.796121 26818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:18.754484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51440 (local address 127.15.75.188:40911)
0809 02:16:18.754742 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:18.754748 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:18.785584 (+ 30836us) server_negotiation.cc:407] Connection header received
0809 02:16:18.785688 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:18.785692 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:18.785766 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:18.785859 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:18.786801 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:18.787685 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:18.788998 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:18.789357 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:18.789465 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:18.791971 (+ 2506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:18.791975 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:18.791979 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:18.792010 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:18.794353 (+ 2343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:18.794825 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:18.794827 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:18.794829 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:18.794903 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:18.795190 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:18.795195 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:18.795197 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:18.795382 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:18.795520 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:18.795833 (+ 313us) server_negotiation.cc:299] Negotiation successful
0809 02:16:18.795950 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":102,"threads_started":1}
I20260809 02:16:18.803663 26818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:18.793440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51450 (local address 127.15.75.188:40911)
0809 02:16:18.796275 (+ 2835us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:18.796279 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:18.796298 (+ 19us) server_negotiation.cc:407] Connection header received
0809 02:16:18.796336 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:18.796338 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:18.796370 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:18.796460 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:18.797576 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:18.798067 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:18.798902 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:18.799137 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:18.799246 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:18.800505 (+ 1259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:18.800508 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:18.800510 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:18.800533 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:18.801922 (+ 1389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:18.802399 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:18.802402 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:18.802403 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:18.802475 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:18.802786 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:18.802789 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:18.802791 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:18.802942 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:18.803035 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:18.803330 (+ 295us) server_negotiation.cc:299] Negotiation successful
0809 02:16:18.803431 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2770,"thread_start_us":86,"threads_started":1}
I20260809 02:16:19.739955 26831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:19.739419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:42501 (local address 127.15.75.130:38371)
0809 02:16:19.739752 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:19.739842 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:42501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":70,"threads_started":1}
W20260809 02:16:19.740278 26528 heartbeater.cc:644] Failed to heartbeat to 127.15.75.190:42501 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.15.75.190:42501: connect: Connection refused (error 111)
I20260809 02:16:19.739591 26830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:19.739124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:42501 (local address 127.15.75.131:57961)
0809 02:16:19.739370 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:19.739472 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:42501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":102,"threads_started":1}
I20260809 02:16:19.740712 26832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:19.740130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:42501 (local address 127.15.75.129:35047)
0809 02:16:19.740430 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:19.740522 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:42501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":66,"threads_started":1}
W20260809 02:16:19.740803 26735 heartbeater.cc:644] Failed to heartbeat to 127.15.75.190:42501 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.15.75.190:42501: connect: Connection refused (error 111)
W20260809 02:16:19.740991 26312 heartbeater.cc:644] Failed to heartbeat to 127.15.75.190:42501 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.15.75.190:42501: connect: Connection refused (error 111)
I20260809 02:16:20.251811 26833 raft_consensus.cc:482] T 00000000000000000000000000000000 P a39abb470ce448908175ec3b2e32e19f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c59a0fdc35554648bd22e9b2280fe648)
I20260809 02:16:20.252110 26833 raft_consensus.cc:504] T 00000000000000000000000000000000 P a39abb470ce448908175ec3b2e32e19f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c59a0fdc35554648bd22e9b2280fe648" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 42501 } } peers { permanent_uuid: "b4f88c387e1d4b3c83ef38475e586f91" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 42709 } attrs { promote: false } } peers { permanent_uuid: "a39abb470ce448908175ec3b2e32e19f" member_type: VOTER last_known_addr { host: "127.15.75.188" port: 40911 } attrs { promote: false } }
I20260809 02:16:20.253965 26833 leader_election.cc:288] T 00000000000000000000000000000000 P a39abb470ce448908175ec3b2e32e19f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c59a0fdc35554648bd22e9b2280fe648 (127.15.75.190:42501), b4f88c387e1d4b3c83ef38475e586f91 (127.15.75.189:42709)
I20260809 02:16:20.255235 26836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:20.253725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:42501 (local address 127.0.0.1:60042)
0809 02:16:20.255029 (+ 1304us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:20.255119 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:42501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1109,"thread_start_us":67,"threads_started":1}
W20260809 02:16:20.255416 26012 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.75.190:42501: connect: Connection refused (error 111)
I20260809 02:16:20.256007 26839 raft_consensus.cc:482] T 00000000000000000000000000000000 P b4f88c387e1d4b3c83ef38475e586f91 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c59a0fdc35554648bd22e9b2280fe648)
I20260809 02:16:20.256225 26836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:20.256014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:42501 (local address 127.0.0.1:60056)
0809 02:16:20.256103 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:20.256155 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:42501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260809 02:16:20.256168 26839 raft_consensus.cc:504] T 00000000000000000000000000000000 P b4f88c387e1d4b3c83ef38475e586f91 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c59a0fdc35554648bd22e9b2280fe648" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 42501 } } peers { permanent_uuid: "b4f88c387e1d4b3c83ef38475e586f91" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 42709 } attrs { promote: false } } peers { permanent_uuid: "a39abb470ce448908175ec3b2e32e19f" member_type: VOTER last_known_addr { host: "127.15.75.188" port: 40911 } attrs { promote: false } }
W20260809 02:16:20.256328 26012 leader_election.cc:334] T 00000000000000000000000000000000 P a39abb470ce448908175ec3b2e32e19f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c59a0fdc35554648bd22e9b2280fe648 (127.15.75.190:42501): Network error: Client connection negotiation failed: client connection to 127.15.75.190:42501: connect: Connection refused (error 111)
I20260809 02:16:20.257143 26839 leader_election.cc:288] T 00000000000000000000000000000000 P b4f88c387e1d4b3c83ef38475e586f91 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c59a0fdc35554648bd22e9b2280fe648 (127.15.75.190:42501), a39abb470ce448908175ec3b2e32e19f (127.15.75.188:40911)
I20260809 02:16:20.257643 26840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:20.257203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:42501 (local address 127.0.0.1:60060)
0809 02:16:20.257370 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:20.257545 (+ 175us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:42501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":57,"threads_started":1}
W20260809 02:16:20.257923 25734 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.75.190:42501: connect: Connection refused (error 111)
I20260809 02:16:20.258882 26840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:20.258673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:42501 (local address 127.0.0.1:60072)
0809 02:16:20.258782 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:20.258830 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:42501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260809 02:16:20.259032 25734 leader_election.cc:334] T 00000000000000000000000000000000 P b4f88c387e1d4b3c83ef38475e586f91 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c59a0fdc35554648bd22e9b2280fe648 (127.15.75.190:42501): Network error: Client connection negotiation failed: client connection to 127.15.75.190:42501: connect: Connection refused (error 111)
I20260809 02:16:20.260865 26834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:20.253747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.189:42709 (local address 127.0.0.1:41224)
0809 02:16:20.255048 (+ 1301us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:20.255063 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:20.255161 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:20.256197 (+ 1036us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:20.256201 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:20.256221 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:16:20.256501 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:20.256510 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:20.257564 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:20.257568 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:20.259326 (+ 1758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:20.259336 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:20.260432 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:20.260435 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:20.260529 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:20.260551 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 02:16:20.260626 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 02:16:20.260722 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1247,"thread_start_us":78,"threads_started":1}
I20260809 02:16:20.260921 26837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:20.254448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41224 (local address 127.15.75.189:42709)
0809 02:16:20.255562 (+ 1114us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:20.255567 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:20.255580 (+ 13us) server_negotiation.cc:407] Connection header received
0809 02:16:20.255630 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:20.255632 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:20.255669 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:20.255990 (+ 321us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:16:20.256615 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:20.257407 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:20.259468 (+ 2061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:20.260200 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:20.260302 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:20.260360 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:20.260661 (+ 301us) server_negotiation.cc:299] Negotiation successful
0809 02:16:20.260781 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":78,"server-negotiator.queue_time_us":1030,"thread_start_us":82,"threads_started":1}
I20260809 02:16:20.261467 25766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a39abb470ce448908175ec3b2e32e19f" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "b4f88c387e1d4b3c83ef38475e586f91" is_pre_election: true
I20260809 02:16:20.261659 25766 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b4f88c387e1d4b3c83ef38475e586f91 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a39abb470ce448908175ec3b2e32e19f in term 1.
I20260809 02:16:20.262019 26010 leader_election.cc:302] T 00000000000000000000000000000000 P a39abb470ce448908175ec3b2e32e19f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a39abb470ce448908175ec3b2e32e19f, b4f88c387e1d4b3c83ef38475e586f91; no voters: c59a0fdc35554648bd22e9b2280fe648
I20260809 02:16:20.262104 26841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:20.257527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.188:40911 (local address 127.0.0.1:51458)
0809 02:16:20.257735 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:20.257760 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:20.257843 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:20.258315 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:20.258318 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:20.258336 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:16:20.258438 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:20.258449 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:20.259424 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:20.259426 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:20.260748 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:20.260759 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:20.261694 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:20.261698 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:20.261786 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:20.261806 (+ 20us) client_negotiation.cc:769] Sending connection context
0809 02:16:20.261898 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 02:16:20.261967 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"mutex_wait_us":56,"thread_start_us":76,"threads_started":1}
I20260809 02:16:20.262137 26842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:20.257868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51458 (local address 127.15.75.188:40911)
0809 02:16:20.258110 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:20.258120 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:20.258133 (+ 13us) server_negotiation.cc:407] Connection header received
0809 02:16:20.258175 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:20.258178 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:20.258216 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:20.258303 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:16:20.258569 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:20.259309 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:20.260877 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:20.261520 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:20.261691 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:20.261745 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:20.261916 (+ 171us) server_negotiation.cc:299] Negotiation successful
0809 02:16:20.261983 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":109,"threads_started":1}
I20260809 02:16:20.262415 26833 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a39abb470ce448908175ec3b2e32e19f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 02:16:20.262482 26833 raft_consensus.cc:482] T 00000000000000000000000000000000 P a39abb470ce448908175ec3b2e32e19f [term 1 FOLLOWER]: Starting leader election (detected failure of leader c59a0fdc35554648bd22e9b2280fe648)
I20260809 02:16:20.262523 26833 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a39abb470ce448908175ec3b2e32e19f [term 1 FOLLOWER]: Advancing to term 2
I20260809 02:16:20.262542 26047 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b4f88c387e1d4b3c83ef38475e586f91" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "a39abb470ce448908175ec3b2e32e19f" is_pre_election: true
I20260809 02:16:20.263594 26833 raft_consensus.cc:504] T 00000000000000000000000000000000 P a39abb470ce448908175ec3b2e32e19f [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c59a0fdc35554648bd22e9b2280fe648" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 42501 } } peers { permanent_uuid: "b4f88c387e1d4b3c83ef38475e586f91" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 42709 } attrs { promote: false } } peers { permanent_uuid: "a39abb470ce448908175ec3b2e32e19f" member_type: VOTER last_known_addr { host: "127.15.75.188" port: 40911 } attrs { promote: false } }
I20260809 02:16:20.263835 26047 raft_consensus.cc:2379] T 00000000000000000000000000000000 P a39abb470ce448908175ec3b2e32e19f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f88c387e1d4b3c83ef38475e586f91 in current term 2: Already voted for candidate a39abb470ce448908175ec3b2e32e19f in this term.
I20260809 02:16:20.264297 26834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:20.264076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:42501 (local address 127.0.0.1:60076)
0809 02:16:20.264198 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:20.264251 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:42501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260809 02:16:20.264475 26833 leader_election.cc:288] T 00000000000000000000000000000000 P a39abb470ce448908175ec3b2e32e19f [CANDIDATE]: Term 2 election: Requested vote from peers c59a0fdc35554648bd22e9b2280fe648 (127.15.75.190:42501), b4f88c387e1d4b3c83ef38475e586f91 (127.15.75.189:42709)
I20260809 02:16:20.264739 25766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a39abb470ce448908175ec3b2e32e19f" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "b4f88c387e1d4b3c83ef38475e586f91"
I20260809 02:16:20.264743 25734 leader_election.cc:302] T 00000000000000000000000000000000 P b4f88c387e1d4b3c83ef38475e586f91 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f88c387e1d4b3c83ef38475e586f91; no voters: a39abb470ce448908175ec3b2e32e19f, c59a0fdc35554648bd22e9b2280fe648
I20260809 02:16:20.264858 25766 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b4f88c387e1d4b3c83ef38475e586f91 [term 1 FOLLOWER]: Advancing to term 2
I20260809 02:16:20.265014 26834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:20.264793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:42501 (local address 127.0.0.1:60084)
0809 02:16:20.264905 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:20.264959 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:42501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260809 02:16:20.265156 26012 leader_election.cc:334] T 00000000000000000000000000000000 P a39abb470ce448908175ec3b2e32e19f [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer c59a0fdc35554648bd22e9b2280fe648 (127.15.75.190:42501): Network error: Client connection negotiation failed: client connection to 127.15.75.190:42501: connect: Connection refused (error 111)
I20260809 02:16:20.265825 25766 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b4f88c387e1d4b3c83ef38475e586f91 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a39abb470ce448908175ec3b2e32e19f in term 2.
I20260809 02:16:20.265973 26839 raft_consensus.cc:2726] T 00000000000000000000000000000000 P b4f88c387e1d4b3c83ef38475e586f91 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 02:16:20.266152 26010 leader_election.cc:302] T 00000000000000000000000000000000 P a39abb470ce448908175ec3b2e32e19f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a39abb470ce448908175ec3b2e32e19f, b4f88c387e1d4b3c83ef38475e586f91; no voters: c59a0fdc35554648bd22e9b2280fe648
I20260809 02:16:20.266319 26833 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a39abb470ce448908175ec3b2e32e19f [term 2 FOLLOWER]: Leader election won for term 2
I20260809 02:16:20.266526 26833 raft_consensus.cc:686] T 00000000000000000000000000000000 P a39abb470ce448908175ec3b2e32e19f [term 2 LEADER]: Becoming Leader. State: Replica: a39abb470ce448908175ec3b2e32e19f, State: Running, Role: LEADER
I20260809 02:16:20.266770 26833 consensus_queue.cc:237] T 00000000000000000000000000000000 P a39abb470ce448908175ec3b2e32e19f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c59a0fdc35554648bd22e9b2280fe648" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 42501 } } peers { permanent_uuid: "b4f88c387e1d4b3c83ef38475e586f91" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 42709 } attrs { promote: false } } peers { permanent_uuid: "a39abb470ce448908175ec3b2e32e19f" member_type: VOTER last_known_addr { host: "127.15.75.188" port: 40911 } attrs { promote: false } }
I20260809 02:16:20.268043 26845 sys_catalog.cc:455] T 00000000000000000000000000000000 P a39abb470ce448908175ec3b2e32e19f [sys.catalog]: SysCatalogTable state changed. Reason: New leader a39abb470ce448908175ec3b2e32e19f. Latest consensus state: current_term: 2 leader_uuid: "a39abb470ce448908175ec3b2e32e19f" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c59a0fdc35554648bd22e9b2280fe648" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 42501 } } peers { permanent_uuid: "b4f88c387e1d4b3c83ef38475e586f91" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 42709 } attrs { promote: false } } peers { permanent_uuid: "a39abb470ce448908175ec3b2e32e19f" member_type: VOTER last_known_addr { host: "127.15.75.188" port: 40911 } attrs { promote: false } } }
I20260809 02:16:20.268172 26845 sys_catalog.cc:458] T 00000000000000000000000000000000 P a39abb470ce448908175ec3b2e32e19f [sys.catalog]: This master's current role is: LEADER
I20260809 02:16:20.268463 26847 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:16:20.270803 26847 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:16:20.271195 26847 catalog_manager.cc:1259] Loaded cluster ID: 60284658a9354acb9a8cbfb97c102def
I20260809 02:16:20.271241 26847 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:16:20.271776 26847 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:16:20.272171 26847 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a39abb470ce448908175ec3b2e32e19f: Loaded TSK: 0
I20260809 02:16:20.272903 26847 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:16:20.312928 26024 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51450:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:16:20.317039 26024 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:16:20.320329 26834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:20.320082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:42501 (local address 127.0.0.1:60096)
0809 02:16:20.320186 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:20.320246 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:42501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260809 02:16:20.320387 25766 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b4f88c387e1d4b3c83ef38475e586f91 [term 2 FOLLOWER]: Refusing update from remote peer a39abb470ce448908175ec3b2e32e19f: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260809 02:16:20.320896 26833 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a39abb470ce448908175ec3b2e32e19f [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f88c387e1d4b3c83ef38475e586f91" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 42709 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260809 02:16:20.321076 26834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:20.320817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:42501 (local address 127.0.0.1:60110)
0809 02:16:20.320959 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:20.321005 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:42501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
W20260809 02:16:20.321177 26012 consensus_peers.cc:487] T 00000000000000000000000000000000 P a39abb470ce448908175ec3b2e32e19f -> Peer c59a0fdc35554648bd22e9b2280fe648 (127.15.75.190:42501): Couldn't send request to peer c59a0fdc35554648bd22e9b2280fe648. Status: Network error: Client connection negotiation failed: client connection to 127.15.75.190:42501: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 02:16:20.324061 26839 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4f88c387e1d4b3c83ef38475e586f91 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a39abb470ce448908175ec3b2e32e19f. Latest consensus state: current_term: 2 leader_uuid: "a39abb470ce448908175ec3b2e32e19f" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c59a0fdc35554648bd22e9b2280fe648" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 42501 } } peers { permanent_uuid: "b4f88c387e1d4b3c83ef38475e586f91" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 42709 } attrs { promote: false } } peers { permanent_uuid: "a39abb470ce448908175ec3b2e32e19f" member_type: VOTER last_known_addr { host: "127.15.75.188" port: 40911 } attrs { promote: false } } }
I20260809 02:16:20.324213 26839 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4f88c387e1d4b3c83ef38475e586f91 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:16:20.325246 26833 sys_catalog.cc:455] T 00000000000000000000000000000000 P a39abb470ce448908175ec3b2e32e19f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "a39abb470ce448908175ec3b2e32e19f" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c59a0fdc35554648bd22e9b2280fe648" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 42501 } } peers { permanent_uuid: "b4f88c387e1d4b3c83ef38475e586f91" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 42709 } attrs { promote: false } } peers { permanent_uuid: "a39abb470ce448908175ec3b2e32e19f" member_type: VOTER last_known_addr { host: "127.15.75.188" port: 40911 } attrs { promote: false } } }
I20260809 02:16:20.325367 26833 sys_catalog.cc:458] T 00000000000000000000000000000000 P a39abb470ce448908175ec3b2e32e19f [sys.catalog]: This master's current role is: LEADER
I20260809 02:16:20.326490 26839 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4f88c387e1d4b3c83ef38475e586f91 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "a39abb470ce448908175ec3b2e32e19f" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c59a0fdc35554648bd22e9b2280fe648" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 42501 } } peers { permanent_uuid: "b4f88c387e1d4b3c83ef38475e586f91" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 42709 } attrs { promote: false } } peers { permanent_uuid: "a39abb470ce448908175ec3b2e32e19f" member_type: VOTER last_known_addr { host: "127.15.75.188" port: 40911 } attrs { promote: false } } }
I20260809 02:16:20.326598 26839 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4f88c387e1d4b3c83ef38475e586f91 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:16:20.327145 26849 mvcc.cc:205] Tried to move back new op lower bound from 7316446332187398144 to 7316446331974881280. Current Snapshot: MvccSnapshot[applied={T|T < 7316446332187398144}]
I20260809 02:16:20.337102 26834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:20.331907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.131:35531 (local address 127.0.0.1:35488)
0809 02:16:20.332023 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:20.332040 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:20.332115 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:20.333343 (+ 1228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:20.333346 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:20.333363 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:16:20.333529 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:20.333539 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:20.334604 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:20.334607 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:20.335762 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:20.335771 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:20.336724 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:20.336728 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:20.336818 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:20.336845 (+ 27us) client_negotiation.cc:769] Sending connection context
0809 02:16:20.336891 (+ 46us) client_negotiation.cc:240] Negotiation successful
0809 02:16:20.336958 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28,"mutex_wait_us":1}
I20260809 02:16:20.337791 26856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:20.332560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35488 (local address 127.15.75.131:35531)
0809 02:16:20.333042 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:20.333046 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:20.333058 (+ 12us) server_negotiation.cc:407] Connection header received
0809 02:16:20.333137 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:20.333140 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:20.333190 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:20.333643 (+ 453us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:16:20.333735 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:20.334479 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:20.335908 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:20.336571 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:20.337499 (+ 928us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:20.337563 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:20.337596 (+ 33us) server_negotiation.cc:299] Negotiation successful
0809 02:16:20.337664 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":93,"threads_started":1}
I20260809 02:16:20.338436 26633 tablet_service.cc:1467] Processing CreateTablet for tablet 4f29d539971d4d1b822996c95f2b289a (DEFAULT_TABLE table=test-table [id=a7780a8d0cd24f21acd8f807fcf2577d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:16:20.338762 26633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f29d539971d4d1b822996c95f2b289a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:16:20.339233 26836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:20.332651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.130:36535 (local address 127.0.0.1:37006)
0809 02:16:20.332826 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:20.332841 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:20.332930 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:20.333696 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:20.333699 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:20.333718 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:16:20.333819 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:20.333825 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:20.335126 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:20.335131 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:20.336600 (+ 1469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:20.336616 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:20.338764 (+ 2148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:20.338769 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:20.338872 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:20.338899 (+ 27us) client_negotiation.cc:769] Sending connection context
0809 02:16:20.338956 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 02:16:20.339058 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"mutex_wait_us":1}
I20260809 02:16:20.339080 26858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:20.333163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.129:34105 (local address 127.0.0.1:51310)
0809 02:16:20.333399 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:20.333411 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:20.333504 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:20.334938 (+ 1434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:20.334941 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:20.334960 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:16:20.335050 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:20.335055 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:20.336044 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:20.336047 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:20.337305 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:20.337315 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:20.338622 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:20.338625 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:20.338734 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:20.338756 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 02:16:20.338872 (+ 116us) client_negotiation.cc:240] Negotiation successful
0809 02:16:20.338954 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"mutex_wait_us":48,"thread_start_us":105,"threads_started":1}
I20260809 02:16:20.339730 26861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:20.333465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51310 (local address 127.15.75.129:34105)
0809 02:16:20.333730 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:20.333734 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:20.333747 (+ 13us) server_negotiation.cc:407] Connection header received
0809 02:16:20.333796 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:20.333799 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:20.333847 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:20.333929 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:16:20.335182 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:20.335928 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:20.337446 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:20.338412 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:20.338515 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:20.339173 (+ 658us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:20.339215 (+ 42us) server_negotiation.cc:299] Negotiation successful
0809 02:16:20.339608 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":117,"threads_started":1}
I20260809 02:16:20.339884 26857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:20.332798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37006 (local address 127.15.75.130:36535)
0809 02:16:20.333038 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:20.333044 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:20.333059 (+ 15us) server_negotiation.cc:407] Connection header received
0809 02:16:20.333117 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:20.333119 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:20.333170 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:20.333267 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:16:20.333958 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:20.334806 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:20.336814 (+ 2008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:20.338605 (+ 1791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:20.339503 (+ 898us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:20.339571 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:20.339615 (+ 44us) server_negotiation.cc:299] Negotiation successful
0809 02:16:20.339711 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":68,"threads_started":1}
I20260809 02:16:20.340370 26424 tablet_service.cc:1467] Processing CreateTablet for tablet 4f29d539971d4d1b822996c95f2b289a (DEFAULT_TABLE table=test-table [id=a7780a8d0cd24f21acd8f807fcf2577d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:16:20.340364 26206 tablet_service.cc:1467] Processing CreateTablet for tablet 4f29d539971d4d1b822996c95f2b289a (DEFAULT_TABLE table=test-table [id=a7780a8d0cd24f21acd8f807fcf2577d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:16:20.340633 26424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f29d539971d4d1b822996c95f2b289a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:16:20.340633 26206 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f29d539971d4d1b822996c95f2b289a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:16:20.341641 26864 tablet_bootstrap.cc:492] T 4f29d539971d4d1b822996c95f2b289a P acf85e824204477bb6d89218e1ed0533: Bootstrap starting.
I20260809 02:16:20.342864 26864 tablet_bootstrap.cc:654] T 4f29d539971d4d1b822996c95f2b289a P acf85e824204477bb6d89218e1ed0533: Neither blocks nor log segments found. Creating new log.
I20260809 02:16:20.343164 26865 tablet_bootstrap.cc:492] T 4f29d539971d4d1b822996c95f2b289a P 6babbe94e6d54368a5c7edf6c5847ba1: Bootstrap starting.
I20260809 02:16:20.343856 26866 tablet_bootstrap.cc:492] T 4f29d539971d4d1b822996c95f2b289a P 6ee9ba5954274e238185dfb3cb465605: Bootstrap starting.
I20260809 02:16:20.344163 26865 tablet_bootstrap.cc:654] T 4f29d539971d4d1b822996c95f2b289a P 6babbe94e6d54368a5c7edf6c5847ba1: Neither blocks nor log segments found. Creating new log.
I20260809 02:16:20.344666 26864 tablet_bootstrap.cc:492] T 4f29d539971d4d1b822996c95f2b289a P acf85e824204477bb6d89218e1ed0533: No bootstrap required, opened a new log
I20260809 02:16:20.344815 26864 ts_tablet_manager.cc:1397] T 4f29d539971d4d1b822996c95f2b289a P acf85e824204477bb6d89218e1ed0533: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.001s
I20260809 02:16:20.344923 26866 tablet_bootstrap.cc:654] T 4f29d539971d4d1b822996c95f2b289a P 6ee9ba5954274e238185dfb3cb465605: Neither blocks nor log segments found. Creating new log.
I20260809 02:16:20.345360 26864 raft_consensus.cc:348] T 4f29d539971d4d1b822996c95f2b289a P acf85e824204477bb6d89218e1ed0533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf85e824204477bb6d89218e1ed0533" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 35531 } } peers { permanent_uuid: "6ee9ba5954274e238185dfb3cb465605" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 36535 } } peers { permanent_uuid: "6babbe94e6d54368a5c7edf6c5847ba1" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 34105 } }
I20260809 02:16:20.345481 26864 raft_consensus.cc:374] T 4f29d539971d4d1b822996c95f2b289a P acf85e824204477bb6d89218e1ed0533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:16:20.345463 26865 tablet_bootstrap.cc:492] T 4f29d539971d4d1b822996c95f2b289a P 6babbe94e6d54368a5c7edf6c5847ba1: No bootstrap required, opened a new log
I20260809 02:16:20.345533 26864 raft_consensus.cc:729] T 4f29d539971d4d1b822996c95f2b289a P acf85e824204477bb6d89218e1ed0533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acf85e824204477bb6d89218e1ed0533, State: Initialized, Role: FOLLOWER
I20260809 02:16:20.345651 26865 ts_tablet_manager.cc:1397] T 4f29d539971d4d1b822996c95f2b289a P 6babbe94e6d54368a5c7edf6c5847ba1: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260809 02:16:20.345683 26864 consensus_queue.cc:260] T 4f29d539971d4d1b822996c95f2b289a P acf85e824204477bb6d89218e1ed0533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf85e824204477bb6d89218e1ed0533" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 35531 } } peers { permanent_uuid: "6ee9ba5954274e238185dfb3cb465605" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 36535 } } peers { permanent_uuid: "6babbe94e6d54368a5c7edf6c5847ba1" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 34105 } }
I20260809 02:16:20.346148 26865 raft_consensus.cc:348] T 4f29d539971d4d1b822996c95f2b289a P 6babbe94e6d54368a5c7edf6c5847ba1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf85e824204477bb6d89218e1ed0533" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 35531 } } peers { permanent_uuid: "6ee9ba5954274e238185dfb3cb465605" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 36535 } } peers { permanent_uuid: "6babbe94e6d54368a5c7edf6c5847ba1" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 34105 } }
I20260809 02:16:20.346304 26865 raft_consensus.cc:374] T 4f29d539971d4d1b822996c95f2b289a P 6babbe94e6d54368a5c7edf6c5847ba1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:16:20.346378 26865 raft_consensus.cc:729] T 4f29d539971d4d1b822996c95f2b289a P 6babbe94e6d54368a5c7edf6c5847ba1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6babbe94e6d54368a5c7edf6c5847ba1, State: Initialized, Role: FOLLOWER
I20260809 02:16:20.346431 26866 tablet_bootstrap.cc:492] T 4f29d539971d4d1b822996c95f2b289a P 6ee9ba5954274e238185dfb3cb465605: No bootstrap required, opened a new log
I20260809 02:16:20.346575 26866 ts_tablet_manager.cc:1397] T 4f29d539971d4d1b822996c95f2b289a P 6ee9ba5954274e238185dfb3cb465605: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 02:16:20.346609 26865 consensus_queue.cc:260] T 4f29d539971d4d1b822996c95f2b289a P 6babbe94e6d54368a5c7edf6c5847ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf85e824204477bb6d89218e1ed0533" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 35531 } } peers { permanent_uuid: "6ee9ba5954274e238185dfb3cb465605" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 36535 } } peers { permanent_uuid: "6babbe94e6d54368a5c7edf6c5847ba1" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 34105 } }
I20260809 02:16:20.347025 26865 ts_tablet_manager.cc:1428] T 4f29d539971d4d1b822996c95f2b289a P 6babbe94e6d54368a5c7edf6c5847ba1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260809 02:16:20.347164 26866 raft_consensus.cc:348] T 4f29d539971d4d1b822996c95f2b289a P 6ee9ba5954274e238185dfb3cb465605 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf85e824204477bb6d89218e1ed0533" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 35531 } } peers { permanent_uuid: "6ee9ba5954274e238185dfb3cb465605" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 36535 } } peers { permanent_uuid: "6babbe94e6d54368a5c7edf6c5847ba1" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 34105 } }
I20260809 02:16:20.347277 26866 raft_consensus.cc:374] T 4f29d539971d4d1b822996c95f2b289a P 6ee9ba5954274e238185dfb3cb465605 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:16:20.347316 26866 raft_consensus.cc:729] T 4f29d539971d4d1b822996c95f2b289a P 6ee9ba5954274e238185dfb3cb465605 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ee9ba5954274e238185dfb3cb465605, State: Initialized, Role: FOLLOWER
I20260809 02:16:20.348553 26866 consensus_queue.cc:260] T 4f29d539971d4d1b822996c95f2b289a P 6ee9ba5954274e238185dfb3cb465605 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf85e824204477bb6d89218e1ed0533" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 35531 } } peers { permanent_uuid: "6ee9ba5954274e238185dfb3cb465605" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 36535 } } peers { permanent_uuid: "6babbe94e6d54368a5c7edf6c5847ba1" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 34105 } }
I20260809 02:16:20.349625 26878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:20.349244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:42501 (local address 127.15.75.130:41581)
0809 02:16:20.349475 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:20.349541 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:42501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":107,"threads_started":1}
I20260809 02:16:20.349795 26881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:20.349275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:42501 (local address 127.15.75.129:36941)
0809 02:16:20.349544 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:20.349604 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:42501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":103,"threads_started":1}
I20260809 02:16:20.351003 26881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:20.350788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:42501 (local address 127.15.75.129:42511)
0809 02:16:20.350895 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:20.350943 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:42501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260809 02:16:20.349025 26866 ts_tablet_manager.cc:1428] T 4f29d539971d4d1b822996c95f2b289a P 6ee9ba5954274e238185dfb3cb465605: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260809 02:16:20.351790 26309 heartbeater.cc:503] Master 127.15.75.188:40911 was elected leader, sending a full tablet report...
I20260809 02:16:20.352013 26527 heartbeater.cc:503] Master 127.15.75.188:40911 was elected leader, sending a full tablet report...
I20260809 02:16:20.352620 26878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:20.352418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:42501 (local address 127.15.75.130:51327)
0809 02:16:20.352507 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:20.352559 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:42501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260809 02:16:20.353155 26864 ts_tablet_manager.cc:1428] T 4f29d539971d4d1b822996c95f2b289a P acf85e824204477bb6d89218e1ed0533: Time spent starting tablet: real 0.008s user 0.000s sys 0.001s
I20260809 02:16:20.353344 26733 heartbeater.cc:503] Master 127.15.75.188:40911 was elected leader, sending a full tablet report...
I20260809 02:16:20.354168 26875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:20.346336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:42501 (local address 127.15.75.131:51247)
0809 02:16:20.353993 (+ 7657us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:20.354049 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:42501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":7595,"thread_start_us":94,"threads_started":1}
I20260809 02:16:20.354753 26875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:20.354549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:42501 (local address 127.15.75.131:36687)
0809 02:16:20.354652 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:20.354696 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:42501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260809 02:16:20.355299 26877 raft_consensus.cc:482] T 4f29d539971d4d1b822996c95f2b289a P 6ee9ba5954274e238185dfb3cb465605 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:16:20.355405 26877 raft_consensus.cc:504] T 4f29d539971d4d1b822996c95f2b289a P 6ee9ba5954274e238185dfb3cb465605 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf85e824204477bb6d89218e1ed0533" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 35531 } } peers { permanent_uuid: "6ee9ba5954274e238185dfb3cb465605" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 36535 } } peers { permanent_uuid: "6babbe94e6d54368a5c7edf6c5847ba1" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 34105 } }
I20260809 02:16:20.355857 26877 leader_election.cc:288] T 4f29d539971d4d1b822996c95f2b289a P 6ee9ba5954274e238185dfb3cb465605 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acf85e824204477bb6d89218e1ed0533 (127.15.75.131:35531), 6babbe94e6d54368a5c7edf6c5847ba1 (127.15.75.129:34105)
I20260809 02:16:20.356109 26665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f29d539971d4d1b822996c95f2b289a" candidate_uuid: "6ee9ba5954274e238185dfb3cb465605" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf85e824204477bb6d89218e1ed0533" is_pre_election: true
I20260809 02:16:20.356226 26233 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f29d539971d4d1b822996c95f2b289a" candidate_uuid: "6ee9ba5954274e238185dfb3cb465605" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6babbe94e6d54368a5c7edf6c5847ba1" is_pre_election: true
I20260809 02:16:20.356257 26665 raft_consensus.cc:2454] T 4f29d539971d4d1b822996c95f2b289a P acf85e824204477bb6d89218e1ed0533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ee9ba5954274e238185dfb3cb465605 in term 0.
I20260809 02:16:20.356353 26233 raft_consensus.cc:2454] T 4f29d539971d4d1b822996c95f2b289a P 6babbe94e6d54368a5c7edf6c5847ba1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ee9ba5954274e238185dfb3cb465605 in term 0.
I20260809 02:16:20.356585 26342 leader_election.cc:302] T 4f29d539971d4d1b822996c95f2b289a P 6ee9ba5954274e238185dfb3cb465605 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ee9ba5954274e238185dfb3cb465605, acf85e824204477bb6d89218e1ed0533; no voters:
I20260809 02:16:20.356758 26877 raft_consensus.cc:2781] T 4f29d539971d4d1b822996c95f2b289a P 6ee9ba5954274e238185dfb3cb465605 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:16:20.356825 26877 raft_consensus.cc:482] T 4f29d539971d4d1b822996c95f2b289a P 6ee9ba5954274e238185dfb3cb465605 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:16:20.356875 26877 raft_consensus.cc:3037] T 4f29d539971d4d1b822996c95f2b289a P 6ee9ba5954274e238185dfb3cb465605 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:16:20.358335 26877 raft_consensus.cc:504] T 4f29d539971d4d1b822996c95f2b289a P 6ee9ba5954274e238185dfb3cb465605 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf85e824204477bb6d89218e1ed0533" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 35531 } } peers { permanent_uuid: "6ee9ba5954274e238185dfb3cb465605" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 36535 } } peers { permanent_uuid: "6babbe94e6d54368a5c7edf6c5847ba1" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 34105 } }
I20260809 02:16:20.358744 26877 leader_election.cc:288] T 4f29d539971d4d1b822996c95f2b289a P 6ee9ba5954274e238185dfb3cb465605 [CANDIDATE]: Term 1 election: Requested vote from peers acf85e824204477bb6d89218e1ed0533 (127.15.75.131:35531), 6babbe94e6d54368a5c7edf6c5847ba1 (127.15.75.129:34105)
I20260809 02:16:20.359014 26665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f29d539971d4d1b822996c95f2b289a" candidate_uuid: "6ee9ba5954274e238185dfb3cb465605" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf85e824204477bb6d89218e1ed0533"
I20260809 02:16:20.359046 26233 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f29d539971d4d1b822996c95f2b289a" candidate_uuid: "6ee9ba5954274e238185dfb3cb465605" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6babbe94e6d54368a5c7edf6c5847ba1"
I20260809 02:16:20.359143 26665 raft_consensus.cc:3037] T 4f29d539971d4d1b822996c95f2b289a P acf85e824204477bb6d89218e1ed0533 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:16:20.359160 26233 raft_consensus.cc:3037] T 4f29d539971d4d1b822996c95f2b289a P 6babbe94e6d54368a5c7edf6c5847ba1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:16:20.359934 26233 raft_consensus.cc:2454] T 4f29d539971d4d1b822996c95f2b289a P 6babbe94e6d54368a5c7edf6c5847ba1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ee9ba5954274e238185dfb3cb465605 in term 1.
I20260809 02:16:20.360221 26345 leader_election.cc:302] T 4f29d539971d4d1b822996c95f2b289a P 6ee9ba5954274e238185dfb3cb465605 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6babbe94e6d54368a5c7edf6c5847ba1, 6ee9ba5954274e238185dfb3cb465605; no voters:
I20260809 02:16:20.360430 26877 raft_consensus.cc:2781] T 4f29d539971d4d1b822996c95f2b289a P 6ee9ba5954274e238185dfb3cb465605 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:16:20.360416 26665 raft_consensus.cc:2454] T 4f29d539971d4d1b822996c95f2b289a P acf85e824204477bb6d89218e1ed0533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ee9ba5954274e238185dfb3cb465605 in term 1.
I20260809 02:16:20.360636 26877 raft_consensus.cc:686] T 4f29d539971d4d1b822996c95f2b289a P 6ee9ba5954274e238185dfb3cb465605 [term 1 LEADER]: Becoming Leader. State: Replica: 6ee9ba5954274e238185dfb3cb465605, State: Running, Role: LEADER
I20260809 02:16:20.360821 26877 consensus_queue.cc:237] T 4f29d539971d4d1b822996c95f2b289a P 6ee9ba5954274e238185dfb3cb465605 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf85e824204477bb6d89218e1ed0533" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 35531 } } peers { permanent_uuid: "6ee9ba5954274e238185dfb3cb465605" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 36535 } } peers { permanent_uuid: "6babbe94e6d54368a5c7edf6c5847ba1" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 34105 } }
I20260809 02:16:20.361557 26878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:20.361331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:42501 (local address 127.15.75.130:60125)
0809 02:16:20.361433 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:20.361489 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:42501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260809 02:16:20.362921 26021 catalog_manager.cc:5495] T 4f29d539971d4d1b822996c95f2b289a P 6ee9ba5954274e238185dfb3cb465605 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6ee9ba5954274e238185dfb3cb465605 (127.15.75.130). New cstate: current_term: 1 leader_uuid: "6ee9ba5954274e238185dfb3cb465605" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf85e824204477bb6d89218e1ed0533" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 35531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ee9ba5954274e238185dfb3cb465605" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 36535 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6babbe94e6d54368a5c7edf6c5847ba1" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 34105 } health_report { overall_health: UNKNOWN } } }
I20260809 02:16:20.378973 26857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:20.375823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37012 (local address 127.15.75.130:36535)
0809 02:16:20.375956 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:20.375960 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:20.375974 (+ 14us) server_negotiation.cc:407] Connection header received
0809 02:16:20.376061 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:20.376065 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:20.376110 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:20.376195 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:16:20.376481 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:20.377086 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:20.377936 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:20.378166 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:20.378272 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:20.378533 (+ 261us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:16:20.378607 (+ 74us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:16:20.378686 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:20.378775 (+ 89us) server_negotiation.cc:299] Negotiation successful
0809 02:16:20.378828 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 02:16:20.381069 26233 raft_consensus.cc:1261] T 4f29d539971d4d1b822996c95f2b289a P 6babbe94e6d54368a5c7edf6c5847ba1 [term 1 FOLLOWER]: Refusing update from remote peer 6ee9ba5954274e238185dfb3cb465605: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:16:20.381083 26665 raft_consensus.cc:1261] T 4f29d539971d4d1b822996c95f2b289a P acf85e824204477bb6d89218e1ed0533 [term 1 FOLLOWER]: Refusing update from remote peer 6ee9ba5954274e238185dfb3cb465605: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:16:20.381533 26877 consensus_queue.cc:1035] T 4f29d539971d4d1b822996c95f2b289a P 6ee9ba5954274e238185dfb3cb465605 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6babbe94e6d54368a5c7edf6c5847ba1" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 34105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:16:20.381733 26883 consensus_queue.cc:1035] T 4f29d539971d4d1b822996c95f2b289a P 6ee9ba5954274e238185dfb3cb465605 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf85e824204477bb6d89218e1ed0533" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 35531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:16:20.383424 26875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:20.383113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:42501 (local address 127.15.75.131:51153)
0809 02:16:20.383278 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:20.383335 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:42501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260809 02:16:20.384171 26881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:20.383817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:42501 (local address 127.15.75.129:46947)
0809 02:16:20.384044 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:20.384094 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:42501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165}
I20260809 02:16:20.385708 26875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:20.385463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:42501 (local address 127.15.75.131:48237)
0809 02:16:20.385597 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:20.385646 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:42501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260809 02:16:20.387948 26878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:20.386384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:42501 (local address 127.15.75.130:52821)
0809 02:16:20.387827 (+ 1443us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:20.387881 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:42501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1357}
I20260809 02:16:20.389243 26881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:20.388998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:42501 (local address 127.15.75.129:36709)
0809 02:16:20.389127 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:20.389174 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:42501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260809 02:16:20.390112 26878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:20.389853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:42501 (local address 127.15.75.130:34009)
0809 02:16:20.389995 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:20.390047 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:42501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260809 02:16:20.394436 26021 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51450:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:16:20.394598 26021 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51450:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:16:20.398059 26021 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a39abb470ce448908175ec3b2e32e19f: Sending DeleteTablet for 3 replicas of tablet 4f29d539971d4d1b822996c95f2b289a
I20260809 02:16:20.398851 26206 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f29d539971d4d1b822996c95f2b289a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:16:20 UTC) from {username='oryx', principal='oryx/127.15.75.188@KRBTEST.COM'} at 127.0.0.1:51310
I20260809 02:16:20.398888 26633 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f29d539971d4d1b822996c95f2b289a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:16:20 UTC) from {username='oryx', principal='oryx/127.15.75.188@KRBTEST.COM'} at 127.0.0.1:35488
I20260809 02:16:20.399158 26424 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f29d539971d4d1b822996c95f2b289a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:16:20 UTC) from {username='oryx', principal='oryx/127.15.75.188@KRBTEST.COM'} at 127.0.0.1:37006
I20260809 02:16:20.399222 26905 tablet_replica.cc:330] T 4f29d539971d4d1b822996c95f2b289a P acf85e824204477bb6d89218e1ed0533: stopping tablet replica
I20260809 02:16:20.399339 26905 raft_consensus.cc:2229] T 4f29d539971d4d1b822996c95f2b289a P acf85e824204477bb6d89218e1ed0533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:16:20.399437 26906 tablet_replica.cc:330] T 4f29d539971d4d1b822996c95f2b289a P 6ee9ba5954274e238185dfb3cb465605: stopping tablet replica
I20260809 02:16:20.399461 26905 raft_consensus.cc:2258] T 4f29d539971d4d1b822996c95f2b289a P acf85e824204477bb6d89218e1ed0533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:16:20.399554 26906 raft_consensus.cc:2229] T 4f29d539971d4d1b822996c95f2b289a P 6ee9ba5954274e238185dfb3cb465605 [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:16:20.399689 26904 tablet_replica.cc:330] T 4f29d539971d4d1b822996c95f2b289a P 6babbe94e6d54368a5c7edf6c5847ba1: stopping tablet replica
I20260809 02:16:20.399708 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 25969
I20260809 02:16:20.399827 26904 raft_consensus.cc:2229] T 4f29d539971d4d1b822996c95f2b289a P 6babbe94e6d54368a5c7edf6c5847ba1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:16:20.399963 26904 raft_consensus.cc:2258] T 4f29d539971d4d1b822996c95f2b289a P 6babbe94e6d54368a5c7edf6c5847ba1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:16:20.399760 26906 raft_consensus.cc:2258] T 4f29d539971d4d1b822996c95f2b289a P 6ee9ba5954274e238185dfb3cb465605 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:16:20.403155 26905 ts_tablet_manager.cc:1905] T 4f29d539971d4d1b822996c95f2b289a P acf85e824204477bb6d89218e1ed0533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:16:20.404625 26906 ts_tablet_manager.cc:1905] T 4f29d539971d4d1b822996c95f2b289a P 6ee9ba5954274e238185dfb3cb465605: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:16:20.405946 26905 ts_tablet_manager.cc:1918] T 4f29d539971d4d1b822996c95f2b289a P acf85e824204477bb6d89218e1ed0533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:16:20.406085 26905 log.cc:1196] T 4f29d539971d4d1b822996c95f2b289a P acf85e824204477bb6d89218e1ed0533: Deleting WAL directory at /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-2/wal/wals/4f29d539971d4d1b822996c95f2b289a
I20260809 02:16:20.406442 26905 ts_tablet_manager.cc:1939] T 4f29d539971d4d1b822996c95f2b289a P acf85e824204477bb6d89218e1ed0533: Deleting consensus metadata
I20260809 02:16:20.407200 26009 catalog_manager.cc:4841] TS acf85e824204477bb6d89218e1ed0533 (127.15.75.131:35531): tablet 4f29d539971d4d1b822996c95f2b289a (table test-table [id=a7780a8d0cd24f21acd8f807fcf2577d]) successfully deleted
I20260809 02:16:20.407601 26906 ts_tablet_manager.cc:1918] T 4f29d539971d4d1b822996c95f2b289a P 6ee9ba5954274e238185dfb3cb465605: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:16:20.407696 26906 log.cc:1196] T 4f29d539971d4d1b822996c95f2b289a P 6ee9ba5954274e238185dfb3cb465605: Deleting WAL directory at /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241741788589-15662-0/minicluster-data/ts-1/wal/wals/4f29d539971d4d1b822996c95f2b289a
I20260809 02:16:20.407963 26906 ts_tablet_manager.cc:1939] T 4f29d539971d4d1b822996c95f2b289a P 6ee9ba5954274e238185dfb3cb465605: Deleting consensus metadata
I20260809 02:16:20.413801 26009 catalog_manager.cc:4841] TS 6ee9ba5954274e238185dfb3cb465605 (127.15.75.130:36535): tablet 4f29d539971d4d1b822996c95f2b289a (table test-table [id=a7780a8d0cd24f21acd8f807fcf2577d]) successfully deleted
I20260809 02:16:20.419615 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 26323
I20260809 02:16:20.419872 26858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:20.419543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.129:34105 (local address 127.0.0.1:51326)
0809 02:16:20.419700 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:20.419769 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.129:34105: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260809 02:16:20.422670 26010 catalog_manager.cc:4584] TS 6babbe94e6d54368a5c7edf6c5847ba1 (127.15.75.129:34105): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4f29d539971d4d1b822996c95f2b289a: Network error: Client connection negotiation failed: client connection to 127.15.75.129:34105: connect: Connection refused (error 111)
I20260809 02:16:20.431203 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 26535
I20260809 02:16:20.439512 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 25528
I20260809 02:16:20.453562 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 25707
2026-08-09T02:16:20Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (5840 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:16:20 dist-test-slave-s11t krb5kdc[26909](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:16:20 dist-test-slave-s11t krb5kdc[26909](info): set up 2 sockets
Aug 09 02:16:20 dist-test-slave-s11t krb5kdc[26909](info): commencing operation
krb5kdc: starting...
W20260809 02:16:21.702176 15662 mini_kdc.cc:121] Time spent starting KDC: real 1.220s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Aug 09 02:16:21 dist-test-slave-s11t krb5kdc[26909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241781, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:16:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:16:21Z Disabled control of system clock
WARNING: no policy specified for oryx/127.15.75.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.75.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.75.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/krb5kdc/oryx_127.15.75.190.keytab.
Entry for principal oryx/127.15.75.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/krb5kdc/oryx_127.15.75.190.keytab.
I20260809 02:16:21.827344 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:38449
--webserver_interface=127.15.75.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/krb5kdc/oryx_127.15.75.190.keytab
--principal=oryx/127.15.75.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:34131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.75.190:38449
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:16:21.937626 26929 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:16:21.937883 26929 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:16:21.937922 26929 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:16:21.941843 26929 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:16:21.941893 26929 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 02:16:21.941910 26929 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:16:21.941929 26929 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:16:21.941944 26929 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:16:21.941959 26929 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:16:21.947101 26929 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:34131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.75.190:38449
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/krb5kdc/oryx_127.15.75.190.keytab
--principal=oryx/127.15.75.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:38449
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.75.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26929
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:16:21.948015 26929 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:16:21.948961 26929 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:16:21.957240 26937 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:21.957371 26929 server_base.cc:1034] running on GCE node
W20260809 02:16:21.957271 26938 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:21.957255 26944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:21.958062 26929 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:16:21.959041 26929 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:16:21.960201 26929 hybrid_clock.cc:648] HybridClock initialized: now 1786241781960180 us; error 35 us; skew 500 ppm
Aug 09 02:16:21 dist-test-slave-s11t krb5kdc[26909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241781, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.75.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:16:21.963039 26929 init.cc:377] Logged in from keytab as oryx/127.15.75.190@KRBTEST.COM (short username oryx)
I20260809 02:16:21.963899 26929 webserver.cc:460] Webserver started at http://127.15.75.190:39951/ using document root <none> and password file <none>
I20260809 02:16:21.964442 26929 fs_manager.cc:359] Metadata directory not provided
I20260809 02:16:21.964531 26929 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:16:21.964831 26929 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:16:21.966795 26929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/minicluster-data/master-0/data/instance:
uuid: "34bc02e2aabc462baa95d3fc45949cba"
format_stamp: "Formatted at 2026-08-09 02:16:21 on dist-test-slave-s11t"
server_key: "d2a1b4bcfa87e16d2cdb9bd5d157841c"
server_key_iv: "7bce42be00d3c06e7d5d41db01d3cf1d"
server_key_version: "encryptionkey@0"
I20260809 02:16:21.967355 26929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/minicluster-data/master-0/wal/instance:
uuid: "34bc02e2aabc462baa95d3fc45949cba"
format_stamp: "Formatted at 2026-08-09 02:16:21 on dist-test-slave-s11t"
server_key: "d2a1b4bcfa87e16d2cdb9bd5d157841c"
server_key_iv: "7bce42be00d3c06e7d5d41db01d3cf1d"
server_key_version: "encryptionkey@0"
I20260809 02:16:21.970968 26929 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 02:16:21.973300 26954 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:21.974507 26929 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 02:16:21.974650 26929 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/minicluster-data/master-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/minicluster-data/master-0/wal
uuid: "34bc02e2aabc462baa95d3fc45949cba"
format_stamp: "Formatted at 2026-08-09 02:16:21 on dist-test-slave-s11t"
server_key: "d2a1b4bcfa87e16d2cdb9bd5d157841c"
server_key_iv: "7bce42be00d3c06e7d5d41db01d3cf1d"
server_key_version: "encryptionkey@0"
I20260809 02:16:21.974790 26929 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:16:21.983976 26929 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:16:21.986606 26929 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:16:21.986812 26929 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:16:21.994668 26929 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.190:38449
I20260809 02:16:21.994679 27053 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.190:38449 every 8 connection(s)
I20260809 02:16:21.995549 26929 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
I20260809 02:16:21.998270 27055 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:16:22.003400 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 26929
I20260809 02:16:22.003545 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/minicluster-data/master-0/wal/instance
I20260809 02:16:22.003696 27055 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34bc02e2aabc462baa95d3fc45949cba: Bootstrap starting.
I20260809 02:16:22.003877 15662 external_mini_cluster.cc:1428] Setting key f88b9e96d0adcb4706f1b1fffb7dae36
I20260809 02:16:22.006260 27055 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 34bc02e2aabc462baa95d3fc45949cba: Neither blocks nor log segments found. Creating new log.
I20260809 02:16:22.007012 27055 log.cc:824] T 00000000000000000000000000000000 P 34bc02e2aabc462baa95d3fc45949cba: Log is configured to *not* fsync() on all Append() calls
I20260809 02:16:22.008890 27055 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34bc02e2aabc462baa95d3fc45949cba: No bootstrap required, opened a new log
Aug 09 02:16:22 dist-test-slave-s11t krb5kdc[26909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241781, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM
I20260809 02:16:22.012610 27055 raft_consensus.cc:348] T 00000000000000000000000000000000 P 34bc02e2aabc462baa95d3fc45949cba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34bc02e2aabc462baa95d3fc45949cba" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 38449 } }
I20260809 02:16:22.012921 27055 raft_consensus.cc:374] T 00000000000000000000000000000000 P 34bc02e2aabc462baa95d3fc45949cba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:16:22.013005 27055 raft_consensus.cc:729] T 00000000000000000000000000000000 P 34bc02e2aabc462baa95d3fc45949cba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34bc02e2aabc462baa95d3fc45949cba, State: Initialized, Role: FOLLOWER
I20260809 02:16:22.013478 27055 consensus_queue.cc:260] T 00000000000000000000000000000000 P 34bc02e2aabc462baa95d3fc45949cba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34bc02e2aabc462baa95d3fc45949cba" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 38449 } }
I20260809 02:16:22.013666 27055 raft_consensus.cc:388] T 00000000000000000000000000000000 P 34bc02e2aabc462baa95d3fc45949cba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:16:22.013743 27055 raft_consensus.cc:482] T 00000000000000000000000000000000 P 34bc02e2aabc462baa95d3fc45949cba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:16:22.013890 27055 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 34bc02e2aabc462baa95d3fc45949cba [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:16:22.015120 27055 raft_consensus.cc:504] T 00000000000000000000000000000000 P 34bc02e2aabc462baa95d3fc45949cba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34bc02e2aabc462baa95d3fc45949cba" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 38449 } }
I20260809 02:16:22.015491 27055 leader_election.cc:302] T 00000000000000000000000000000000 P 34bc02e2aabc462baa95d3fc45949cba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 34bc02e2aabc462baa95d3fc45949cba; no voters:
I20260809 02:16:22.015825 27055 leader_election.cc:288] T 00000000000000000000000000000000 P 34bc02e2aabc462baa95d3fc45949cba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:16:22.015939 27061 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 34bc02e2aabc462baa95d3fc45949cba [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:16:22.016188 27061 raft_consensus.cc:686] T 00000000000000000000000000000000 P 34bc02e2aabc462baa95d3fc45949cba [term 1 LEADER]: Becoming Leader. State: Replica: 34bc02e2aabc462baa95d3fc45949cba, State: Running, Role: LEADER
I20260809 02:16:22.016498 27061 consensus_queue.cc:237] T 00000000000000000000000000000000 P 34bc02e2aabc462baa95d3fc45949cba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34bc02e2aabc462baa95d3fc45949cba" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 38449 } }
I20260809 02:16:22.017163 27055 sys_catalog.cc:564] T 00000000000000000000000000000000 P 34bc02e2aabc462baa95d3fc45949cba [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:16:22.019462 27059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:22.005310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48544 (local address 127.15.75.190:38449)
0809 02:16:22.005736 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:22.005751 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:22.005786 (+ 35us) server_negotiation.cc:407] Connection header received
0809 02:16:22.006480 (+ 694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:22.006499 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:22.006816 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:22.007156 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:22.007991 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:22.009080 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:22.009924 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:22.010377 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:22.010553 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:22.013401 (+ 2848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:22.013407 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:22.013422 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:22.013451 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:22.015583 (+ 2132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:22.016127 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:22.016133 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:22.016139 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:22.016331 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:22.016663 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:22.016666 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:22.016667 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:22.017183 (+ 516us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:22.017415 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:22.018078 (+ 663us) server_negotiation.cc:299] Negotiation successful
0809 02:16:22.018403 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":126,"threads_started":1}
I20260809 02:16:22.020180 27062 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34bc02e2aabc462baa95d3fc45949cba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "34bc02e2aabc462baa95d3fc45949cba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34bc02e2aabc462baa95d3fc45949cba" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 38449 } } }
I20260809 02:16:22.020344 27062 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34bc02e2aabc462baa95d3fc45949cba [sys.catalog]: This master's current role is: LEADER
I20260809 02:16:22.020551 27063 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34bc02e2aabc462baa95d3fc45949cba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 34bc02e2aabc462baa95d3fc45949cba. Latest consensus state: current_term: 1 leader_uuid: "34bc02e2aabc462baa95d3fc45949cba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34bc02e2aabc462baa95d3fc45949cba" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 38449 } } }
I20260809 02:16:22.020812 27063 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34bc02e2aabc462baa95d3fc45949cba [sys.catalog]: This master's current role is: LEADER
W20260809 02:16:22.022835 27087 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 34bc02e2aabc462baa95d3fc45949cba: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:16:22.022925 27087 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:16:22.022992 27089 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:16:22.023696 27089 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:16:22.028673 27089 catalog_manager.cc:1347] Generated new cluster ID: d89db5a087da4dcc94351fcb0aa66b15
I20260809 02:16:22.028755 27089 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:16:22.047269 27089 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:16:22.048130 27089 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:16:22.055374 27089 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 34bc02e2aabc462baa95d3fc45949cba: Generated new TSK 0
I20260809 02:16:22.055975 27089 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.15.75.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.75.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.75.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/krb5kdc/oryx_127.15.75.129.keytab.
Entry for principal oryx/127.15.75.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/krb5kdc/oryx_127.15.75.129.keytab.
I20260809 02:16:22.091387 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.129:0
--local_ip_for_outbound_sockets=127.15.75.129
--webserver_interface=127.15.75.129
--webserver_port=0
--tserver_master_addrs=127.15.75.190:38449
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/krb5kdc/oryx_127.15.75.129.keytab
--principal=oryx/127.15.75.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:34131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:16:22.204104 27095 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:16:22.204355 27095 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:16:22.204397 27095 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:16:22.208372 27095 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:16:22.208425 27095 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:16:22.208462 27095 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 02:16:22.208525 27095 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.129
I20260809 02:16:22.213723 27095 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:34131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/krb5kdc/oryx_127.15.75.129.keytab
--principal=oryx/127.15.75.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.75.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.75.190:38449
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27095
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.129
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:16:22.214715 27095 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:16:22.215713 27095 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:16:22.225999 27105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:22.226038 27102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:22.226013 27103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:22.226291 27095 server_base.cc:1034] running on GCE node
I20260809 02:16:22.226797 27095 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:16:22.227420 27095 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:16:22.228600 27095 hybrid_clock.cc:648] HybridClock initialized: now 1786241782228572 us; error 53 us; skew 500 ppm
Aug 09 02:16:22 dist-test-slave-s11t krb5kdc[26909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241782, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.75.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:16:22.231376 27095 init.cc:377] Logged in from keytab as oryx/127.15.75.129@KRBTEST.COM (short username oryx)
I20260809 02:16:22.232213 27095 webserver.cc:460] Webserver started at http://127.15.75.129:35325/ using document root <none> and password file <none>
I20260809 02:16:22.232842 27095 fs_manager.cc:359] Metadata directory not provided
I20260809 02:16:22.232937 27095 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:16:22.233150 27095 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:16:22.235128 27095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/minicluster-data/ts-0/data/instance:
uuid: "a1a992abbf44400bbf100e73e082e99f"
format_stamp: "Formatted at 2026-08-09 02:16:22 on dist-test-slave-s11t"
server_key: "fea9cdb8eda1964d5d7385a0c8a968f6"
server_key_iv: "7054cbc5f5d0cf365e9a44a9341e5758"
server_key_version: "encryptionkey@0"
I20260809 02:16:22.235724 27095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/minicluster-data/ts-0/wal/instance:
uuid: "a1a992abbf44400bbf100e73e082e99f"
format_stamp: "Formatted at 2026-08-09 02:16:22 on dist-test-slave-s11t"
server_key: "fea9cdb8eda1964d5d7385a0c8a968f6"
server_key_iv: "7054cbc5f5d0cf365e9a44a9341e5758"
server_key_version: "encryptionkey@0"
I20260809 02:16:22.239605 27095 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260809 02:16:22.242040 27115 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:22.243307 27095 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 02:16:22.243485 27095 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/minicluster-data/ts-0/wal
uuid: "a1a992abbf44400bbf100e73e082e99f"
format_stamp: "Formatted at 2026-08-09 02:16:22 on dist-test-slave-s11t"
server_key: "fea9cdb8eda1964d5d7385a0c8a968f6"
server_key_iv: "7054cbc5f5d0cf365e9a44a9341e5758"
server_key_version: "encryptionkey@0"
I20260809 02:16:22.243606 27095 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:16:22.279613 27095 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:16:22.283247 27095 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:16:22.283488 27095 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:16:22.285166 27095 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:16:22.285244 27095 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:22.285348 27095 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:16:22.285393 27095 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 02:16:22 dist-test-slave-s11t krb5kdc[26909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241782, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.75.129@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM
I20260809 02:16:22.297744 27095 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.129:41883
I20260809 02:16:22.298671 27095 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
I20260809 02:16:22.299643 27286 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.129:41883 every 8 connection(s)
I20260809 02:16:22.300951 27059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:22.285950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.129:36175 (local address 127.15.75.190:38449)
0809 02:16:22.286091 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:22.286095 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:22.287213 (+ 1118us) server_negotiation.cc:407] Connection header received
0809 02:16:22.288396 (+ 1183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:22.288400 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:22.288466 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:22.288532 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:22.290140 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:22.290982 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:22.292492 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:22.292835 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:22.292966 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:22.296750 (+ 3784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:22.296755 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:22.296758 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:22.296788 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:22.298601 (+ 1813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:22.299173 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:22.299180 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:22.299186 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:22.299314 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:22.299793 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:22.299798 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:22.299800 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:22.300024 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:22.300120 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:22.300602 (+ 482us) server_negotiation.cc:299] Negotiation successful
0809 02:16:22.300771 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 02:16:22.301730 27127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:22.286493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:38449 (local address 127.15.75.129:36175)
0809 02:16:22.287034 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:22.287079 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:22.288101 (+ 1022us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:22.289036 (+ 935us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:22.289045 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:22.289467 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:22.289964 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:22.289978 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:22.291146 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:22.291150 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:22.292356 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:22.292367 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:22.293037 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:22.293041 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:22.293228 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:22.293853 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:22.293893 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:22.296539 (+ 2646us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:16:22.298730 (+ 2191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:22.298736 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:22.298750 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:22.299057 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:22.299438 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:22.299443 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:22.299445 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:22.299630 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:22.300135 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:16:22.300140 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:16:22.300376 (+ 236us) client_negotiation.cc:769] Sending connection context
0809 02:16:22.300545 (+ 169us) client_negotiation.cc:240] Negotiation successful
0809 02:16:22.300886 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":150,"threads_started":1}
I20260809 02:16:22.303113 27289 heartbeater.cc:347] Connected to a master server at 127.15.75.190:38449
I20260809 02:16:22.303252 27289 heartbeater.cc:464] Registering TS with master...
I20260809 02:16:22.303853 27289 heartbeater.cc:511] Master 127.15.75.190:38449 requested a full tablet report, sending...
I20260809 02:16:22.305596 26975 ts_manager.cc:194] Registered new tserver with Master: a1a992abbf44400bbf100e73e082e99f (127.15.75.129:41883)
I20260809 02:16:22.306759 26975 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.75.129@KRBTEST.COM'} at 127.15.75.129:36175
I20260809 02:16:22.308389 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 27095
I20260809 02:16:22.308521 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/minicluster-data/ts-0/wal/instance
I20260809 02:16:22.308910 15662 external_mini_cluster.cc:1428] Setting key d483e792c78bbc677759af8ae28342dc
I20260809 02:16:22.313375 27059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:22.305560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.129:55309 (local address 127.15.75.190:38449)
0809 02:16:22.305684 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:22.305688 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:22.305840 (+ 152us) server_negotiation.cc:407] Connection header received
0809 02:16:22.306003 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:22.306006 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:22.306062 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:22.306144 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:22.306553 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:22.307135 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:22.308263 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:22.308607 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:22.308757 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:22.309873 (+ 1116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:22.309877 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:22.309880 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:22.309903 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:22.311606 (+ 1703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:22.312043 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:22.312049 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:22.312053 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:22.312147 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:22.312455 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:22.312461 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:22.312466 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:22.312745 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:22.312856 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:22.313176 (+ 320us) server_negotiation.cc:299] Negotiation successful
0809 02:16:22.313267 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 02:16:22.313562 27295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:22.305448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:38449 (local address 127.15.75.129:55309)
0809 02:16:22.305745 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:22.305773 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:22.305905 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:22.306159 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:22.306162 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:22.306354 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:22.306445 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:22.306451 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:22.307270 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:22.307274 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:22.308129 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:22.308140 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:22.308765 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:22.308769 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:22.308861 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:22.309355 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:22.309371 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:22.309733 (+ 362us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:16:22.311726 (+ 1993us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:22.311729 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:22.311731 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:22.311940 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:22.312248 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:22.312250 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:22.312252 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:22.312352 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:22.312874 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:16:22.312879 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:16:22.312995 (+ 116us) client_negotiation.cc:769] Sending connection context
0809 02:16:22.313125 (+ 130us) client_negotiation.cc:240] Negotiation successful
0809 02:16:22.313305 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":108,"threads_started":1}
WARNING: no policy specified for oryx/127.15.75.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.75.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.75.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/krb5kdc/oryx_127.15.75.130.keytab.
Entry for principal oryx/127.15.75.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/krb5kdc/oryx_127.15.75.130.keytab.
I20260809 02:16:22.339946 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.130:0
--local_ip_for_outbound_sockets=127.15.75.130
--webserver_interface=127.15.75.130
--webserver_port=0
--tserver_master_addrs=127.15.75.190:38449
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/krb5kdc/oryx_127.15.75.130.keytab
--principal=oryx/127.15.75.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:34131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:16:22.454103 27303 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:16:22.454340 27303 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:16:22.454388 27303 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:16:22.458966 27303 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:16:22.459021 27303 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:16:22.459059 27303 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 02:16:22.459090 27303 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.130
I20260809 02:16:22.464378 27303 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:34131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/krb5kdc/oryx_127.15.75.130.keytab
--principal=oryx/127.15.75.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.75.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.75.190:38449
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27303
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.130
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:16:22.465380 27303 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:16:22.466310 27303 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:16:22.475756 27308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:22.475756 27312 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:22.476076 27303 server_base.cc:1034] running on GCE node
W20260809 02:16:22.475829 27309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:22.476485 27303 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:16:22.477144 27303 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:16:22.478318 27303 hybrid_clock.cc:648] HybridClock initialized: now 1786241782478297 us; error 31 us; skew 500 ppm
Aug 09 02:16:22 dist-test-slave-s11t krb5kdc[26909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241782, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.75.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:16:22.481166 27303 init.cc:377] Logged in from keytab as oryx/127.15.75.130@KRBTEST.COM (short username oryx)
I20260809 02:16:22.482035 27303 webserver.cc:460] Webserver started at http://127.15.75.130:46385/ using document root <none> and password file <none>
I20260809 02:16:22.482583 27303 fs_manager.cc:359] Metadata directory not provided
I20260809 02:16:22.482687 27303 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:16:22.482926 27303 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:16:22.484948 27303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/minicluster-data/ts-1/data/instance:
uuid: "e33d0f78bb944d67ac12cec28e0bec5b"
format_stamp: "Formatted at 2026-08-09 02:16:22 on dist-test-slave-s11t"
server_key: "45cbb4cb3290966fdf2a01c9f6ebcee6"
server_key_iv: "08ab7e430bd5df49e70e77538e1599e1"
server_key_version: "encryptionkey@0"
I20260809 02:16:22.485518 27303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/minicluster-data/ts-1/wal/instance:
uuid: "e33d0f78bb944d67ac12cec28e0bec5b"
format_stamp: "Formatted at 2026-08-09 02:16:22 on dist-test-slave-s11t"
server_key: "45cbb4cb3290966fdf2a01c9f6ebcee6"
server_key_iv: "08ab7e430bd5df49e70e77538e1599e1"
server_key_version: "encryptionkey@0"
I20260809 02:16:22.489317 27303 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 02:16:22.491891 27327 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:22.493248 27303 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 02:16:22.493402 27303 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/minicluster-data/ts-1/wal
uuid: "e33d0f78bb944d67ac12cec28e0bec5b"
format_stamp: "Formatted at 2026-08-09 02:16:22 on dist-test-slave-s11t"
server_key: "45cbb4cb3290966fdf2a01c9f6ebcee6"
server_key_iv: "08ab7e430bd5df49e70e77538e1599e1"
server_key_version: "encryptionkey@0"
I20260809 02:16:22.493520 27303 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:16:22.509467 27303 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:16:22.512424 27303 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:16:22.512638 27303 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:16:22.514153 27303 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:16:22.514242 27303 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:22.514338 27303 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:16:22.514394 27303 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 02:16:22 dist-test-slave-s11t krb5kdc[26909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241782, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.75.130@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM
I20260809 02:16:22.527436 27303 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.130:44881
I20260809 02:16:22.527480 27502 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.130:44881 every 8 connection(s)
I20260809 02:16:22.528393 27303 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
I20260809 02:16:22.530563 27059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:22.515143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.130:41653 (local address 127.15.75.190:38449)
0809 02:16:22.515291 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:22.515295 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:22.516456 (+ 1161us) server_negotiation.cc:407] Connection header received
0809 02:16:22.517772 (+ 1316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:22.517777 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:22.517849 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:22.517920 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:22.519885 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:22.520880 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:22.522118 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:22.522389 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:22.522515 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:22.526357 (+ 3842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:22.526361 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:22.526365 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:22.526396 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:22.528136 (+ 1740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:22.528794 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:22.528798 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:22.528800 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:22.528935 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:22.529391 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:22.529394 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:22.529396 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:22.529595 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:22.529711 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:22.530242 (+ 531us) server_negotiation.cc:299] Negotiation successful
0809 02:16:22.530377 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 02:16:22.531332 27340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:22.515715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:38449 (local address 127.15.75.130:41653)
0809 02:16:22.516295 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:22.516349 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:22.517486 (+ 1137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:22.518469 (+ 983us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:22.518484 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:22.519015 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:22.519725 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:22.519742 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:22.521011 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:22.521015 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:22.521977 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:22.521988 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:22.522570 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:22.522575 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:22.522796 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:22.523401 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:22.523428 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:22.526173 (+ 2745us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:16:22.528291 (+ 2118us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:22.528297 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:22.528311 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:22.528656 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:22.529064 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:22.529068 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:22.529070 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:22.529248 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:22.529713 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:16:22.529721 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:16:22.529977 (+ 256us) client_negotiation.cc:769] Sending connection context
0809 02:16:22.530170 (+ 193us) client_negotiation.cc:240] Negotiation successful
0809 02:16:22.530515 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":149,"threads_started":1}
I20260809 02:16:22.532545 27503 heartbeater.cc:347] Connected to a master server at 127.15.75.190:38449
I20260809 02:16:22.532766 27503 heartbeater.cc:464] Registering TS with master...
I20260809 02:16:22.533349 27503 heartbeater.cc:511] Master 127.15.75.190:38449 requested a full tablet report, sending...
I20260809 02:16:22.534466 26977 ts_manager.cc:194] Registered new tserver with Master: e33d0f78bb944d67ac12cec28e0bec5b (127.15.75.130:44881)
I20260809 02:16:22.535320 26977 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.75.130@KRBTEST.COM'} at 127.15.75.130:41653
I20260809 02:16:22.536675 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 27303
I20260809 02:16:22.536885 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/minicluster-data/ts-1/wal/instance
I20260809 02:16:22.537253 15662 external_mini_cluster.cc:1428] Setting key 6fe19ee118babc45f5002be3dcc1e4cc
I20260809 02:16:22.543534 27512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:22.534954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:38449 (local address 127.15.75.130:59295)
0809 02:16:22.535263 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:22.535279 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:22.535427 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:22.535731 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:22.535735 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:22.536006 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:22.536135 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:22.536142 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:22.537271 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:22.537276 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:22.538198 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:22.538209 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:22.538840 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:22.538844 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:22.538959 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:22.539451 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:22.539463 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:22.539774 (+ 311us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:16:22.541907 (+ 2133us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:22.541910 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:22.541912 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:22.542123 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:22.542468 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:22.542470 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:22.542471 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:22.542552 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:22.543126 (+ 574us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:16:22.543128 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:16:22.543205 (+ 77us) client_negotiation.cc:769] Sending connection context
0809 02:16:22.543292 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 02:16:22.543403 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"mutex_wait_us":7,"thread_start_us":103,"threads_started":1}
I20260809 02:16:22.543794 27059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:22.535072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.130:59295 (local address 127.15.75.190:38449)
0809 02:16:22.535202 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:22.535206 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:22.535391 (+ 185us) server_negotiation.cc:407] Connection header received
0809 02:16:22.535539 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:22.535541 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:22.535605 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:22.535692 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:22.536264 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:22.537116 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:22.538325 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:22.538687 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:22.538826 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:22.539930 (+ 1104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:22.539934 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:22.539937 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:22.539969 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:22.541777 (+ 1808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:22.542247 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:22.542253 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:22.542257 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:22.542362 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:22.542652 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:22.542659 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:22.542664 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:22.543015 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:22.543137 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:22.543387 (+ 250us) server_negotiation.cc:299] Negotiation successful
0809 02:16:22.543533 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
WARNING: no policy specified for oryx/127.15.75.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.75.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.75.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/krb5kdc/oryx_127.15.75.131.keytab.
Entry for principal oryx/127.15.75.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/krb5kdc/oryx_127.15.75.131.keytab.
I20260809 02:16:22.567911 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.131:0
--local_ip_for_outbound_sockets=127.15.75.131
--webserver_interface=127.15.75.131
--webserver_port=0
--tserver_master_addrs=127.15.75.190:38449
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/krb5kdc/oryx_127.15.75.131.keytab
--principal=oryx/127.15.75.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:34131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:16:22.683764 27519 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:16:22.684007 27519 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:16:22.684049 27519 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:16:22.687963 27519 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:16:22.688019 27519 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:16:22.688055 27519 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 02:16:22.688087 27519 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.131
I20260809 02:16:22.693125 27519 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:34131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/krb5kdc/oryx_127.15.75.131.keytab
--principal=oryx/127.15.75.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.75.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.75.190:38449
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27519
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.131
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:16:22.694126 27519 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:16:22.695092 27519 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:16:22.704401 27529 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:22.704432 27532 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:22.704432 27530 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:22.704888 27519 server_base.cc:1034] running on GCE node
I20260809 02:16:22.705291 27519 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:16:22.705853 27519 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:16:22.707033 27519 hybrid_clock.cc:648] HybridClock initialized: now 1786241782706994 us; error 53 us; skew 500 ppm
Aug 09 02:16:22 dist-test-slave-s11t krb5kdc[26909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241782, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.75.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:16:22.709903 27519 init.cc:377] Logged in from keytab as oryx/127.15.75.131@KRBTEST.COM (short username oryx)
I20260809 02:16:22.710763 27519 webserver.cc:460] Webserver started at http://127.15.75.131:39217/ using document root <none> and password file <none>
I20260809 02:16:22.711349 27519 fs_manager.cc:359] Metadata directory not provided
I20260809 02:16:22.711447 27519 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:16:22.711694 27519 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:16:22.713935 27519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/minicluster-data/ts-2/data/instance:
uuid: "621578d59c654ad6b768bf8ff3fd7209"
format_stamp: "Formatted at 2026-08-09 02:16:22 on dist-test-slave-s11t"
server_key: "029b2d35e1cd44c1b44ec6de07ca4cec"
server_key_iv: "9603baa04e4152367d2be6396d5be5ee"
server_key_version: "encryptionkey@0"
I20260809 02:16:22.714533 27519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/minicluster-data/ts-2/wal/instance:
uuid: "621578d59c654ad6b768bf8ff3fd7209"
format_stamp: "Formatted at 2026-08-09 02:16:22 on dist-test-slave-s11t"
server_key: "029b2d35e1cd44c1b44ec6de07ca4cec"
server_key_iv: "9603baa04e4152367d2be6396d5be5ee"
server_key_version: "encryptionkey@0"
I20260809 02:16:22.718802 27519 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 02:16:22.721293 27541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:22.722481 27519 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 02:16:22.722633 27519 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/minicluster-data/ts-2/wal
uuid: "621578d59c654ad6b768bf8ff3fd7209"
format_stamp: "Formatted at 2026-08-09 02:16:22 on dist-test-slave-s11t"
server_key: "029b2d35e1cd44c1b44ec6de07ca4cec"
server_key_iv: "9603baa04e4152367d2be6396d5be5ee"
server_key_version: "encryptionkey@0"
I20260809 02:16:22.722766 27519 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:16:22.733330 27519 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:16:22.736307 27519 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:16:22.736603 27519 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:16:22.738013 27519 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:16:22.738090 27519 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:22.738155 27519 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:16:22.738206 27519 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 02:16:22 dist-test-slave-s11t krb5kdc[26909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241782, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.75.131@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM
I20260809 02:16:22.750056 27519 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.131:38591
I20260809 02:16:22.750129 27729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.131:38591 every 8 connection(s)
I20260809 02:16:22.751200 27519 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
I20260809 02:16:22.753911 27059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:22.738902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.131:53179 (local address 127.15.75.190:38449)
0809 02:16:22.739042 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:22.739046 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:22.740194 (+ 1148us) server_negotiation.cc:407] Connection header received
0809 02:16:22.741437 (+ 1243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:22.741443 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:22.741507 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:22.741572 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:22.743514 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:22.744454 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:22.746498 (+ 2044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:22.746770 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:22.746936 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:22.749944 (+ 3008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:22.749951 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:22.749956 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:22.749992 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:22.751683 (+ 1691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:22.752307 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:22.752310 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:22.752312 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:22.752403 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:22.752814 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:22.752817 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:22.752819 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:22.753016 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:22.753129 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:22.753612 (+ 483us) server_negotiation.cc:299] Negotiation successful
0809 02:16:22.753735 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 02:16:22.754717 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 27519
I20260809 02:16:22.754755 27558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:22.739522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:38449 (local address 127.15.75.131:53179)
0809 02:16:22.740030 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:22.740073 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:22.741149 (+ 1076us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:22.742054 (+ 905us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:22.742063 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:22.742557 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:22.743296 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:22.743313 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:22.744650 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:22.744657 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:22.746325 (+ 1668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:22.746340 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:22.746986 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:22.746992 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:22.747204 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:22.747951 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:22.747976 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:22.749761 (+ 1785us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:16:22.751839 (+ 2078us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:22.751847 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:22.751863 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:22.752183 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:22.752499 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:22.752503 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:22.752505 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:22.752649 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:22.753129 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:16:22.753137 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:16:22.753371 (+ 234us) client_negotiation.cc:769] Sending connection context
0809 02:16:22.753547 (+ 176us) client_negotiation.cc:240] Negotiation successful
0809 02:16:22.753860 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":137,"threads_started":1}
I20260809 02:16:22.754892 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/minicluster-data/ts-2/wal/instance
I20260809 02:16:22.755311 15662 external_mini_cluster.cc:1428] Setting key 28b1071fcbe76eeb9e64ecf42de066c6
I20260809 02:16:22.756176 27731 heartbeater.cc:347] Connected to a master server at 127.15.75.190:38449
I20260809 02:16:22.756366 27731 heartbeater.cc:464] Registering TS with master...
I20260809 02:16:22.757026 27731 heartbeater.cc:511] Master 127.15.75.190:38449 requested a full tablet report, sending...
I20260809 02:16:22.758092 26975 ts_manager.cc:194] Registered new tserver with Master: 621578d59c654ad6b768bf8ff3fd7209 (127.15.75.131:38591)
I20260809 02:16:22.758836 26975 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.75.131@KRBTEST.COM'} at 127.15.75.131:53179
I20260809 02:16:22.765503 27739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:22.758473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:38449 (local address 127.15.75.131:48891)
0809 02:16:22.758845 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:22.758858 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:22.758983 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:22.759277 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:22.759280 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:22.759529 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:22.759648 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:22.759656 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:22.760423 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:22.760427 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:22.761223 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:22.761233 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:22.761637 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:22.761640 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:22.761742 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:22.762144 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:22.762156 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:22.762454 (+ 298us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:16:22.764121 (+ 1667us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:22.764123 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:22.764125 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:22.764316 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:22.764580 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:22.764582 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:22.764584 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:22.764672 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:22.765110 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:16:22.765112 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:16:22.765190 (+ 78us) client_negotiation.cc:769] Sending connection context
0809 02:16:22.765282 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 02:16:22.765381 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":105,"threads_started":1}
I20260809 02:16:22.765527 27059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:22.758563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.131:48891 (local address 127.15.75.190:38449)
0809 02:16:22.758703 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:22.758711 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:22.758937 (+ 226us) server_negotiation.cc:407] Connection header received
0809 02:16:22.759090 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:22.759093 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:22.759146 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:22.759258 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:22.759809 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:22.760279 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:22.761334 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:22.761537 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:22.761628 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:22.762578 (+ 950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:22.762580 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:22.762582 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:22.762604 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:22.763998 (+ 1394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:22.764404 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:22.764406 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:22.764407 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:22.764476 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:22.764842 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:22.764844 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:22.764846 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:22.764999 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:22.765078 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:22.765338 (+ 260us) server_negotiation.cc:299] Negotiation successful
0809 02:16:22.765425 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 02:16:22.769338 15662 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 02:16:22 dist-test-slave-s11t krb5kdc[26909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241782, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 02:16:22 dist-test-slave-s11t krb5kdc[26909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:16:22 dist-test-slave-s11t krb5kdc[26909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
W20260809 02:16:22.796178 27059 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:16:22.789834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48556 (local address 127.15.75.190:38449)
0809 02:16:22.789977 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:22.789981 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:22.789996 (+ 15us) server_negotiation.cc:407] Connection header received
0809 02:16:22.790038 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:22.790041 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:22.790081 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:22.790156 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:22.790899 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:22.791425 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:22.792281 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:22.792488 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:22.792565 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:22.796034 (+ 3469us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48556: BlockingRecv error: recv got EOF from 127.0.0.1:48556 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 09 02:16:22 dist-test-slave-s11t krb5kdc[26909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241782, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM
I20260809 02:16:22.805982 27059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:22.797728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48562 (local address 127.15.75.190:38449)
0809 02:16:22.797871 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:22.797875 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:22.797970 (+ 95us) server_negotiation.cc:407] Connection header received
0809 02:16:22.798125 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:22.798132 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:22.798171 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:22.798237 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:22.798811 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:22.799279 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:22.800103 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:22.800320 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:22.800396 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:22.802658 (+ 2262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:22.802661 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:22.802663 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:22.802696 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:22.804190 (+ 1494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:22.804663 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:22.804667 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:22.804668 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:22.804799 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:22.805081 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:22.805085 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:22.805086 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:22.805272 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:22.805414 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:22.805692 (+ 278us) server_negotiation.cc:299] Negotiation successful
0809 02:16:22.805804 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 02:16:22.809144 26975 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48562:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:16:22.812306 26975 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:16:22.828348 27770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:22.821378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.129:41883 (local address 127.0.0.1:40740)
0809 02:16:22.822458 (+ 1080us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:22.822522 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:22.822650 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:22.823297 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:22.823303 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:22.823327 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:16:22.823472 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:22.823480 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:22.825066 (+ 1586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:22.825070 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:22.826760 (+ 1690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:22.826772 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:22.827921 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:22.827925 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:22.828018 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:22.828043 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 02:16:22.828096 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 02:16:22.828165 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":797,"mutex_wait_us":67,"thread_start_us":106,"threads_started":1}
I20260809 02:16:22.828711 27771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:22.821382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57046 (local address 127.15.75.131:38591)
0809 02:16:22.822104 (+ 722us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:22.822117 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:22.822899 (+ 782us) server_negotiation.cc:407] Connection header received
0809 02:16:22.822986 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:22.822992 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:22.823181 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:22.823334 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:16:22.823906 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:22.825143 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:22.826844 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:22.827524 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:22.827655 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:22.827855 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:22.828268 (+ 413us) server_negotiation.cc:299] Negotiation successful
0809 02:16:22.828390 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":615,"thread_start_us":113,"threads_started":1}
I20260809 02:16:22.829017 27772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:22.821888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40740 (local address 127.15.75.129:41883)
0809 02:16:22.822778 (+ 890us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:22.822788 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:22.822816 (+ 28us) server_negotiation.cc:407] Connection header received
0809 02:16:22.822902 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:22.822908 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:22.823127 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:22.823305 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:16:22.823608 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:22.824924 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:22.826918 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:22.827754 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:22.828540 (+ 786us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:22.828717 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:22.828769 (+ 52us) server_negotiation.cc:299] Negotiation successful
0809 02:16:22.828862 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":740,"thread_start_us":90,"threads_started":1}
I20260809 02:16:22.829145 27773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:22.822243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.130:44881 (local address 127.0.0.1:50802)
0809 02:16:22.822537 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:22.822554 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:22.822656 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:22.823486 (+ 830us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:22.823489 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:22.823521 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:16:22.823768 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:22.823773 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:22.825852 (+ 2079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:22.825857 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:22.827049 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:22.827059 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:22.828636 (+ 1577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:22.828639 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:22.828755 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:22.828778 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 02:16:22.828936 (+ 158us) client_negotiation.cc:240] Negotiation successful
0809 02:16:22.828999 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"mutex_wait_us":84,"spinlock_wait_cycles":5888,"thread_start_us":149,"threads_started":1}
I20260809 02:16:22.829778 27774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:22.822418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50802 (local address 127.15.75.130:44881)
0809 02:16:22.822966 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:22.822978 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:22.822997 (+ 19us) server_negotiation.cc:407] Connection header received
0809 02:16:22.823060 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:22.823064 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:22.823204 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:22.823333 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:16:22.824781 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:22.825722 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:22.827183 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:22.828274 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:22.828389 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:22.829504 (+ 1115us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:22.829553 (+ 49us) server_negotiation.cc:299] Negotiation successful
0809 02:16:22.829634 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":99,"threads_started":1}
I20260809 02:16:22.829944 27768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:22.821165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.131:38591 (local address 127.0.0.1:57046)
0809 02:16:22.822696 (+ 1531us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:22.822711 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:22.822794 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:22.823486 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:22.823489 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:22.823520 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:16:22.823765 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:22.823773 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:22.825308 (+ 1535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:22.825311 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:22.826702 (+ 1391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:22.826712 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:22.827654 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:22.827658 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:22.827792 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:22.827841 (+ 49us) client_negotiation.cc:769] Sending connection context
0809 02:16:22.827997 (+ 156us) client_negotiation.cc:240] Negotiation successful
0809 02:16:22.828137 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"mutex_wait_us":176,"spinlock_wait_cycles":4864,"thread_start_us":94,"threads_started":1}
I20260809 02:16:22.832103 27400 tablet_service.cc:1467] Processing CreateTablet for tablet 9e21eb5c46c742c48adaba3408f00bac (DEFAULT_TABLE table=test-table [id=788329e7f82c48e6ab1f220f142dc02e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:16:22.832079 27191 tablet_service.cc:1467] Processing CreateTablet for tablet 9e21eb5c46c742c48adaba3408f00bac (DEFAULT_TABLE table=test-table [id=788329e7f82c48e6ab1f220f142dc02e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:16:22.832569 27626 tablet_service.cc:1467] Processing CreateTablet for tablet 9e21eb5c46c742c48adaba3408f00bac (DEFAULT_TABLE table=test-table [id=788329e7f82c48e6ab1f220f142dc02e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:16:22.833179 27400 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e21eb5c46c742c48adaba3408f00bac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:16:22.833189 27191 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e21eb5c46c742c48adaba3408f00bac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:16:22.833472 27626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e21eb5c46c742c48adaba3408f00bac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:16:22.839952 27775 tablet_bootstrap.cc:492] T 9e21eb5c46c742c48adaba3408f00bac P e33d0f78bb944d67ac12cec28e0bec5b: Bootstrap starting.
I20260809 02:16:22.841004 27777 tablet_bootstrap.cc:492] T 9e21eb5c46c742c48adaba3408f00bac P a1a992abbf44400bbf100e73e082e99f: Bootstrap starting.
I20260809 02:16:22.842655 27775 tablet_bootstrap.cc:654] T 9e21eb5c46c742c48adaba3408f00bac P e33d0f78bb944d67ac12cec28e0bec5b: Neither blocks nor log segments found. Creating new log.
I20260809 02:16:22.843047 27777 tablet_bootstrap.cc:654] T 9e21eb5c46c742c48adaba3408f00bac P a1a992abbf44400bbf100e73e082e99f: Neither blocks nor log segments found. Creating new log.
I20260809 02:16:22.843475 27775 log.cc:824] T 9e21eb5c46c742c48adaba3408f00bac P e33d0f78bb944d67ac12cec28e0bec5b: Log is configured to *not* fsync() on all Append() calls
I20260809 02:16:22.843773 27777 log.cc:824] T 9e21eb5c46c742c48adaba3408f00bac P a1a992abbf44400bbf100e73e082e99f: Log is configured to *not* fsync() on all Append() calls
I20260809 02:16:22.845335 27775 tablet_bootstrap.cc:492] T 9e21eb5c46c742c48adaba3408f00bac P e33d0f78bb944d67ac12cec28e0bec5b: No bootstrap required, opened a new log
I20260809 02:16:22.845345 27777 tablet_bootstrap.cc:492] T 9e21eb5c46c742c48adaba3408f00bac P a1a992abbf44400bbf100e73e082e99f: No bootstrap required, opened a new log
I20260809 02:16:22.845541 27777 ts_tablet_manager.cc:1397] T 9e21eb5c46c742c48adaba3408f00bac P a1a992abbf44400bbf100e73e082e99f: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 02:16:22.845534 27775 ts_tablet_manager.cc:1397] T 9e21eb5c46c742c48adaba3408f00bac P e33d0f78bb944d67ac12cec28e0bec5b: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 02:16:22.848850 27775 raft_consensus.cc:348] T 9e21eb5c46c742c48adaba3408f00bac P e33d0f78bb944d67ac12cec28e0bec5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "621578d59c654ad6b768bf8ff3fd7209" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 38591 } } peers { permanent_uuid: "a1a992abbf44400bbf100e73e082e99f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 41883 } } peers { permanent_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 44881 } }
I20260809 02:16:22.849058 27775 raft_consensus.cc:374] T 9e21eb5c46c742c48adaba3408f00bac P e33d0f78bb944d67ac12cec28e0bec5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:16:22.849125 27775 raft_consensus.cc:729] T 9e21eb5c46c742c48adaba3408f00bac P e33d0f78bb944d67ac12cec28e0bec5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e33d0f78bb944d67ac12cec28e0bec5b, State: Initialized, Role: FOLLOWER
I20260809 02:16:22.849342 27777 raft_consensus.cc:348] T 9e21eb5c46c742c48adaba3408f00bac P a1a992abbf44400bbf100e73e082e99f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "621578d59c654ad6b768bf8ff3fd7209" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 38591 } } peers { permanent_uuid: "a1a992abbf44400bbf100e73e082e99f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 41883 } } peers { permanent_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 44881 } }
I20260809 02:16:22.849587 27777 raft_consensus.cc:374] T 9e21eb5c46c742c48adaba3408f00bac P a1a992abbf44400bbf100e73e082e99f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:16:22.849522 27775 consensus_queue.cc:260] T 9e21eb5c46c742c48adaba3408f00bac P e33d0f78bb944d67ac12cec28e0bec5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "621578d59c654ad6b768bf8ff3fd7209" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 38591 } } peers { permanent_uuid: "a1a992abbf44400bbf100e73e082e99f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 41883 } } peers { permanent_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 44881 } }
I20260809 02:16:22.849653 27777 raft_consensus.cc:729] T 9e21eb5c46c742c48adaba3408f00bac P a1a992abbf44400bbf100e73e082e99f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1a992abbf44400bbf100e73e082e99f, State: Initialized, Role: FOLLOWER
I20260809 02:16:22.850054 27777 consensus_queue.cc:260] T 9e21eb5c46c742c48adaba3408f00bac P a1a992abbf44400bbf100e73e082e99f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "621578d59c654ad6b768bf8ff3fd7209" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 38591 } } peers { permanent_uuid: "a1a992abbf44400bbf100e73e082e99f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 41883 } } peers { permanent_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 44881 } }
I20260809 02:16:22.850237 27503 heartbeater.cc:503] Master 127.15.75.190:38449 was elected leader, sending a full tablet report...
I20260809 02:16:22.850556 27775 ts_tablet_manager.cc:1428] T 9e21eb5c46c742c48adaba3408f00bac P e33d0f78bb944d67ac12cec28e0bec5b: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260809 02:16:22.850605 27289 heartbeater.cc:503] Master 127.15.75.190:38449 was elected leader, sending a full tablet report...
I20260809 02:16:22.844177 27780 tablet_bootstrap.cc:492] T 9e21eb5c46c742c48adaba3408f00bac P 621578d59c654ad6b768bf8ff3fd7209: Bootstrap starting.
I20260809 02:16:22.850939 27777 ts_tablet_manager.cc:1428] T 9e21eb5c46c742c48adaba3408f00bac P a1a992abbf44400bbf100e73e082e99f: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260809 02:16:22.852819 27780 tablet_bootstrap.cc:654] T 9e21eb5c46c742c48adaba3408f00bac P 621578d59c654ad6b768bf8ff3fd7209: Neither blocks nor log segments found. Creating new log.
I20260809 02:16:22.853581 27780 log.cc:824] T 9e21eb5c46c742c48adaba3408f00bac P 621578d59c654ad6b768bf8ff3fd7209: Log is configured to *not* fsync() on all Append() calls
I20260809 02:16:22.856178 27780 tablet_bootstrap.cc:492] T 9e21eb5c46c742c48adaba3408f00bac P 621578d59c654ad6b768bf8ff3fd7209: No bootstrap required, opened a new log
I20260809 02:16:22.856387 27780 ts_tablet_manager.cc:1397] T 9e21eb5c46c742c48adaba3408f00bac P 621578d59c654ad6b768bf8ff3fd7209: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.003s
I20260809 02:16:22.859251 27780 raft_consensus.cc:348] T 9e21eb5c46c742c48adaba3408f00bac P 621578d59c654ad6b768bf8ff3fd7209 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "621578d59c654ad6b768bf8ff3fd7209" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 38591 } } peers { permanent_uuid: "a1a992abbf44400bbf100e73e082e99f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 41883 } } peers { permanent_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 44881 } }
I20260809 02:16:22.859453 27780 raft_consensus.cc:374] T 9e21eb5c46c742c48adaba3408f00bac P 621578d59c654ad6b768bf8ff3fd7209 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:16:22.859534 27780 raft_consensus.cc:729] T 9e21eb5c46c742c48adaba3408f00bac P 621578d59c654ad6b768bf8ff3fd7209 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 621578d59c654ad6b768bf8ff3fd7209, State: Initialized, Role: FOLLOWER
I20260809 02:16:22.859892 27780 consensus_queue.cc:260] T 9e21eb5c46c742c48adaba3408f00bac P 621578d59c654ad6b768bf8ff3fd7209 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "621578d59c654ad6b768bf8ff3fd7209" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 38591 } } peers { permanent_uuid: "a1a992abbf44400bbf100e73e082e99f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 41883 } } peers { permanent_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 44881 } }
I20260809 02:16:22.860504 27731 heartbeater.cc:503] Master 127.15.75.190:38449 was elected leader, sending a full tablet report...
I20260809 02:16:22.860831 27780 ts_tablet_manager.cc:1428] T 9e21eb5c46c742c48adaba3408f00bac P 621578d59c654ad6b768bf8ff3fd7209: Time spent starting tablet: real 0.004s user 0.004s sys 0.001s
I20260809 02:16:22.934551 27787 raft_consensus.cc:482] T 9e21eb5c46c742c48adaba3408f00bac P 621578d59c654ad6b768bf8ff3fd7209 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:16:22.934782 27787 raft_consensus.cc:504] T 9e21eb5c46c742c48adaba3408f00bac P 621578d59c654ad6b768bf8ff3fd7209 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "621578d59c654ad6b768bf8ff3fd7209" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 38591 } } peers { permanent_uuid: "a1a992abbf44400bbf100e73e082e99f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 41883 } } peers { permanent_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 44881 } }
I20260809 02:16:22.935988 27787 leader_election.cc:288] T 9e21eb5c46c742c48adaba3408f00bac P 621578d59c654ad6b768bf8ff3fd7209 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a1a992abbf44400bbf100e73e082e99f (127.15.75.129:41883), e33d0f78bb944d67ac12cec28e0bec5b (127.15.75.130:44881)
I20260809 02:16:22.938478 27785 raft_consensus.cc:482] T 9e21eb5c46c742c48adaba3408f00bac P a1a992abbf44400bbf100e73e082e99f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:16:22.938711 27785 raft_consensus.cc:504] T 9e21eb5c46c742c48adaba3408f00bac P a1a992abbf44400bbf100e73e082e99f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "621578d59c654ad6b768bf8ff3fd7209" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 38591 } } peers { permanent_uuid: "a1a992abbf44400bbf100e73e082e99f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 41883 } } peers { permanent_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 44881 } }
I20260809 02:16:22.940090 27785 leader_election.cc:288] T 9e21eb5c46c742c48adaba3408f00bac P a1a992abbf44400bbf100e73e082e99f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 621578d59c654ad6b768bf8ff3fd7209 (127.15.75.131:38591), e33d0f78bb944d67ac12cec28e0bec5b (127.15.75.130:44881)
I20260809 02:16:22.940920 27774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:22.936432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.131:60821 (local address 127.15.75.130:44881)
0809 02:16:22.936562 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:22.936567 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:22.936965 (+ 398us) server_negotiation.cc:407] Connection header received
0809 02:16:22.937083 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:22.937086 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:22.937141 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:22.937236 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:16:22.937526 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:22.938122 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:22.939625 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:22.940054 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:22.940187 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:22.940250 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:22.940673 (+ 423us) server_negotiation.cc:299] Negotiation successful
0809 02:16:22.940783 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 02:16:22.941562 27789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:22.936321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.130:44881 (local address 127.15.75.131:60821)
0809 02:16:22.936838 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:22.936851 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:22.936962 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:22.937255 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:22.937257 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:22.937272 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:16:22.937403 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:22.937409 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:22.938248 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:22.938251 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:22.939495 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:22.939505 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:22.940369 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:22.940376 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:22.940505 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:22.940543 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 02:16:22.941311 (+ 768us) client_negotiation.cc:240] Negotiation successful
0809 02:16:22.941407 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":429,"mutex_wait_us":94,"thread_start_us":127,"threads_started":1}
I20260809 02:16:22.941803 27558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:22.936321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.129:41883 (local address 127.15.75.131:54653)
0809 02:16:22.936542 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:22.936558 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:22.936662 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:22.937119 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:22.937123 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:22.937143 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:16:22.937302 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:22.937311 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:22.938139 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:22.938145 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:22.940146 (+ 2001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:22.940160 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:22.941274 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:22.941279 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:22.941429 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:22.941455 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 02:16:22.941533 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 02:16:22.941619 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"mutex_wait_us":2}
I20260809 02:16:22.942283 27772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:22.936528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.131:54653 (local address 127.15.75.129:41883)
0809 02:16:22.936645 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:22.936649 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:22.936665 (+ 16us) server_negotiation.cc:407] Connection header received
0809 02:16:22.936873 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:22.936880 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:22.936954 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:22.937034 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:16:22.937434 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:22.938002 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:22.940314 (+ 2312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:22.941017 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:22.941146 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:22.941206 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:22.942057 (+ 851us) server_negotiation.cc:299] Negotiation successful
0809 02:16:22.942159 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 02:16:22.942265 27433 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e21eb5c46c742c48adaba3408f00bac" candidate_uuid: "621578d59c654ad6b768bf8ff3fd7209" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" is_pre_election: true
I20260809 02:16:22.942574 27433 raft_consensus.cc:2454] T 9e21eb5c46c742c48adaba3408f00bac P e33d0f78bb944d67ac12cec28e0bec5b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 621578d59c654ad6b768bf8ff3fd7209 in term 0.
I20260809 02:16:22.942777 27213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e21eb5c46c742c48adaba3408f00bac" candidate_uuid: "621578d59c654ad6b768bf8ff3fd7209" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1a992abbf44400bbf100e73e082e99f" is_pre_election: true
I20260809 02:16:22.942961 27213 raft_consensus.cc:2454] T 9e21eb5c46c742c48adaba3408f00bac P a1a992abbf44400bbf100e73e082e99f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 621578d59c654ad6b768bf8ff3fd7209 in term 0.
I20260809 02:16:22.943202 27548 leader_election.cc:302] T 9e21eb5c46c742c48adaba3408f00bac P 621578d59c654ad6b768bf8ff3fd7209 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 621578d59c654ad6b768bf8ff3fd7209, e33d0f78bb944d67ac12cec28e0bec5b; no voters:
I20260809 02:16:22.943482 27787 raft_consensus.cc:2781] T 9e21eb5c46c742c48adaba3408f00bac P 621578d59c654ad6b768bf8ff3fd7209 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:16:22.943588 27787 raft_consensus.cc:482] T 9e21eb5c46c742c48adaba3408f00bac P 621578d59c654ad6b768bf8ff3fd7209 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:16:22.943639 27787 raft_consensus.cc:3037] T 9e21eb5c46c742c48adaba3408f00bac P 621578d59c654ad6b768bf8ff3fd7209 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:16:22.945102 27787 raft_consensus.cc:504] T 9e21eb5c46c742c48adaba3408f00bac P 621578d59c654ad6b768bf8ff3fd7209 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "621578d59c654ad6b768bf8ff3fd7209" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 38591 } } peers { permanent_uuid: "a1a992abbf44400bbf100e73e082e99f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 41883 } } peers { permanent_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 44881 } }
I20260809 02:16:22.945235 27771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:22.940654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.129:40517 (local address 127.15.75.131:38591)
0809 02:16:22.940778 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:22.940782 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:22.941018 (+ 236us) server_negotiation.cc:407] Connection header received
0809 02:16:22.941215 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:22.941221 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:22.941259 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:22.941301 (+ 42us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:16:22.941703 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:22.942348 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:22.943827 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:22.944523 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:22.944640 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:22.944719 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:22.945060 (+ 341us) server_negotiation.cc:299] Negotiation successful
0809 02:16:22.945137 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 02:16:22.945327 27790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:22.940290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.131:38591 (local address 127.15.75.129:40517)
0809 02:16:22.940935 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:22.940951 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:22.941087 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:22.941423 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:22.941425 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:22.941442 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:16:22.941567 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:22.941577 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:22.942494 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:22.942497 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:22.943691 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:22.943704 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:22.944792 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:22.944796 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:22.944902 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:22.944942 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 02:16:22.945030 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 02:16:22.945121 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":574,"spinlock_wait_cycles":8448,"thread_start_us":127,"threads_started":1}
I20260809 02:16:22.945489 27787 leader_election.cc:288] T 9e21eb5c46c742c48adaba3408f00bac P 621578d59c654ad6b768bf8ff3fd7209 [CANDIDATE]: Term 1 election: Requested vote from peers a1a992abbf44400bbf100e73e082e99f (127.15.75.129:41883), e33d0f78bb944d67ac12cec28e0bec5b (127.15.75.130:44881)
I20260809 02:16:22.945845 27656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e21eb5c46c742c48adaba3408f00bac" candidate_uuid: "a1a992abbf44400bbf100e73e082e99f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "621578d59c654ad6b768bf8ff3fd7209" is_pre_election: true
I20260809 02:16:22.946023 27433 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e21eb5c46c742c48adaba3408f00bac" candidate_uuid: "621578d59c654ad6b768bf8ff3fd7209" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e33d0f78bb944d67ac12cec28e0bec5b"
I20260809 02:16:22.946074 27656 raft_consensus.cc:2379] T 9e21eb5c46c742c48adaba3408f00bac P 621578d59c654ad6b768bf8ff3fd7209 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a1a992abbf44400bbf100e73e082e99f in current term 1: Already voted for candidate 621578d59c654ad6b768bf8ff3fd7209 in this term.
I20260809 02:16:22.946062 27213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e21eb5c46c742c48adaba3408f00bac" candidate_uuid: "621578d59c654ad6b768bf8ff3fd7209" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1a992abbf44400bbf100e73e082e99f"
I20260809 02:16:22.946146 27433 raft_consensus.cc:3037] T 9e21eb5c46c742c48adaba3408f00bac P e33d0f78bb944d67ac12cec28e0bec5b [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:16:22.946199 27213 raft_consensus.cc:3037] T 9e21eb5c46c742c48adaba3408f00bac P a1a992abbf44400bbf100e73e082e99f [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:16:22.946758 27774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:22.940838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.129:37191 (local address 127.15.75.130:44881)
0809 02:16:22.941082 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:22.941086 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:22.941401 (+ 315us) server_negotiation.cc:407] Connection header received
0809 02:16:22.941599 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:22.941602 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:22.941646 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:22.941694 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:16:22.942198 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:22.943307 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:22.945116 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:22.945731 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:22.945834 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:22.945877 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:22.946555 (+ 678us) server_negotiation.cc:299] Negotiation successful
0809 02:16:22.946628 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":125,"threads_started":1}
I20260809 02:16:22.947126 27791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:22.940419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.130:44881 (local address 127.15.75.129:37191)
0809 02:16:22.941275 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:22.941301 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:22.941473 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:22.941845 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:22.941850 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:22.941873 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:16:22.942049 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:22.942056 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:22.943607 (+ 1551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:22.943611 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:22.944974 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:22.944986 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:22.946210 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:22.946235 (+ 25us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:22.946382 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:22.946416 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 02:16:22.946905 (+ 489us) client_negotiation.cc:240] Negotiation successful
0809 02:16:22.946987 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":790,"thread_start_us":127,"threads_started":1}
I20260809 02:16:22.947587 27433 raft_consensus.cc:2454] T 9e21eb5c46c742c48adaba3408f00bac P e33d0f78bb944d67ac12cec28e0bec5b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 621578d59c654ad6b768bf8ff3fd7209 in term 1.
I20260809 02:16:22.947781 27213 raft_consensus.cc:2454] T 9e21eb5c46c742c48adaba3408f00bac P a1a992abbf44400bbf100e73e082e99f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 621578d59c654ad6b768bf8ff3fd7209 in term 1.
I20260809 02:16:22.947985 27548 leader_election.cc:302] T 9e21eb5c46c742c48adaba3408f00bac P 621578d59c654ad6b768bf8ff3fd7209 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 621578d59c654ad6b768bf8ff3fd7209, e33d0f78bb944d67ac12cec28e0bec5b; no voters:
I20260809 02:16:22.948231 27787 raft_consensus.cc:2781] T 9e21eb5c46c742c48adaba3408f00bac P 621578d59c654ad6b768bf8ff3fd7209 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:16:22.948388 27430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e21eb5c46c742c48adaba3408f00bac" candidate_uuid: "a1a992abbf44400bbf100e73e082e99f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" is_pre_election: true
I20260809 02:16:22.948467 27787 raft_consensus.cc:686] T 9e21eb5c46c742c48adaba3408f00bac P 621578d59c654ad6b768bf8ff3fd7209 [term 1 LEADER]: Becoming Leader. State: Replica: 621578d59c654ad6b768bf8ff3fd7209, State: Running, Role: LEADER
I20260809 02:16:22.948601 27430 raft_consensus.cc:2379] T 9e21eb5c46c742c48adaba3408f00bac P e33d0f78bb944d67ac12cec28e0bec5b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a1a992abbf44400bbf100e73e082e99f in current term 1: Already voted for candidate 621578d59c654ad6b768bf8ff3fd7209 in this term.
I20260809 02:16:22.948879 27787 consensus_queue.cc:237] T 9e21eb5c46c742c48adaba3408f00bac P 621578d59c654ad6b768bf8ff3fd7209 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "621578d59c654ad6b768bf8ff3fd7209" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 38591 } } peers { permanent_uuid: "a1a992abbf44400bbf100e73e082e99f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 41883 } } peers { permanent_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 44881 } }
I20260809 02:16:22.950511 27120 leader_election.cc:302] T 9e21eb5c46c742c48adaba3408f00bac P a1a992abbf44400bbf100e73e082e99f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a1a992abbf44400bbf100e73e082e99f; no voters: 621578d59c654ad6b768bf8ff3fd7209, e33d0f78bb944d67ac12cec28e0bec5b
I20260809 02:16:22.950794 27785 raft_consensus.cc:2726] T 9e21eb5c46c742c48adaba3408f00bac P a1a992abbf44400bbf100e73e082e99f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 02:16:22.952145 26975 catalog_manager.cc:5495] T 9e21eb5c46c742c48adaba3408f00bac P 621578d59c654ad6b768bf8ff3fd7209 reported cstate change: term changed from 0 to 1, leader changed from <none> to 621578d59c654ad6b768bf8ff3fd7209 (127.15.75.131). New cstate: current_term: 1 leader_uuid: "621578d59c654ad6b768bf8ff3fd7209" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "621578d59c654ad6b768bf8ff3fd7209" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 38591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a1a992abbf44400bbf100e73e082e99f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 41883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 44881 } health_report { overall_health: UNKNOWN } } }
I20260809 02:16:22.983399 27771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:22.979817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57058 (local address 127.15.75.131:38591)
0809 02:16:22.979964 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:22.979968 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:22.979984 (+ 16us) server_negotiation.cc:407] Connection header received
0809 02:16:22.980062 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:22.980064 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:22.980120 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:22.980213 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:16:22.980441 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:22.981111 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:22.981941 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:22.982220 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:22.982310 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:22.982538 (+ 228us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:16:22.983000 (+ 462us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:16:22.983093 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:22.983152 (+ 59us) server_negotiation.cc:299] Negotiation successful
0809 02:16:22.983200 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 02:16:22.989347 27430 raft_consensus.cc:1261] T 9e21eb5c46c742c48adaba3408f00bac P e33d0f78bb944d67ac12cec28e0bec5b [term 1 FOLLOWER]: Refusing update from remote peer 621578d59c654ad6b768bf8ff3fd7209: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:16:22.989370 27213 raft_consensus.cc:1261] T 9e21eb5c46c742c48adaba3408f00bac P a1a992abbf44400bbf100e73e082e99f [term 1 FOLLOWER]: Refusing update from remote peer 621578d59c654ad6b768bf8ff3fd7209: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:16:22.990099 27794 consensus_queue.cc:1035] T 9e21eb5c46c742c48adaba3408f00bac P 621578d59c654ad6b768bf8ff3fd7209 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 44881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:16:22.990358 27787 consensus_queue.cc:1035] T 9e21eb5c46c742c48adaba3408f00bac P 621578d59c654ad6b768bf8ff3fd7209 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1a992abbf44400bbf100e73e082e99f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 41883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 02:16:23.002216 27733 tablet.cc:2369] T 9e21eb5c46c742c48adaba3408f00bac P 621578d59c654ad6b768bf8ff3fd7209: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:16:23.005065 27800 mvcc.cc:205] Tried to move back new op lower bound from 7316446343116537856 to 7316446342961246208. Current Snapshot: MvccSnapshot[applied={T|T < 7316446343116537856}]
I20260809 02:16:23.014850 26974 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48562:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:16:23.015034 26974 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48562:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:16:23.017658 26974 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 34bc02e2aabc462baa95d3fc45949cba: Sending DeleteTablet for 3 replicas of tablet 9e21eb5c46c742c48adaba3408f00bac
I20260809 02:16:23.018430 27626 tablet_service.cc:1514] Processing DeleteTablet for tablet 9e21eb5c46c742c48adaba3408f00bac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:16:23 UTC) from {username='oryx', principal='oryx/127.15.75.190@KRBTEST.COM'} at 127.0.0.1:57046
I20260809 02:16:23.018664 27400 tablet_service.cc:1514] Processing DeleteTablet for tablet 9e21eb5c46c742c48adaba3408f00bac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:16:23 UTC) from {username='oryx', principal='oryx/127.15.75.190@KRBTEST.COM'} at 127.0.0.1:50802
I20260809 02:16:23.018968 27191 tablet_service.cc:1514] Processing DeleteTablet for tablet 9e21eb5c46c742c48adaba3408f00bac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:16:23 UTC) from {username='oryx', principal='oryx/127.15.75.190@KRBTEST.COM'} at 127.0.0.1:40740
I20260809 02:16:23.019191 27814 tablet_replica.cc:330] T 9e21eb5c46c742c48adaba3408f00bac P e33d0f78bb944d67ac12cec28e0bec5b: stopping tablet replica
I20260809 02:16:23.019548 27814 raft_consensus.cc:2229] T 9e21eb5c46c742c48adaba3408f00bac P e33d0f78bb944d67ac12cec28e0bec5b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:16:23.019594 27813 tablet_replica.cc:330] T 9e21eb5c46c742c48adaba3408f00bac P 621578d59c654ad6b768bf8ff3fd7209: stopping tablet replica
I20260809 02:16:23.019753 26974 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48562:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 02:16:23.019867 27813 raft_consensus.cc:2229] T 9e21eb5c46c742c48adaba3408f00bac P 621578d59c654ad6b768bf8ff3fd7209 [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:16:23.020143 27814 raft_consensus.cc:2258] T 9e21eb5c46c742c48adaba3408f00bac P e33d0f78bb944d67ac12cec28e0bec5b [term 1 FOLLOWER]: Raft consensus is shut down!
W20260809 02:16:23.020169 26974 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:16:23.020220 27813 raft_consensus.cc:2258] T 9e21eb5c46c742c48adaba3408f00bac P 621578d59c654ad6b768bf8ff3fd7209 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:16:23.021811 27816 tablet_replica.cc:330] T 9e21eb5c46c742c48adaba3408f00bac P a1a992abbf44400bbf100e73e082e99f: stopping tablet replica
I20260809 02:16:23.022128 27816 raft_consensus.cc:2229] T 9e21eb5c46c742c48adaba3408f00bac P a1a992abbf44400bbf100e73e082e99f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:16:23.022457 27816 raft_consensus.cc:2258] T 9e21eb5c46c742c48adaba3408f00bac P a1a992abbf44400bbf100e73e082e99f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:16:23.022696 27814 ts_tablet_manager.cc:1905] T 9e21eb5c46c742c48adaba3408f00bac P e33d0f78bb944d67ac12cec28e0bec5b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:16:23.023792 27816 ts_tablet_manager.cc:1905] T 9e21eb5c46c742c48adaba3408f00bac P a1a992abbf44400bbf100e73e082e99f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:16:23.025522 27814 ts_tablet_manager.cc:1918] T 9e21eb5c46c742c48adaba3408f00bac P e33d0f78bb944d67ac12cec28e0bec5b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:16:23.025631 27814 log.cc:1196] T 9e21eb5c46c742c48adaba3408f00bac P e33d0f78bb944d67ac12cec28e0bec5b: Deleting WAL directory at /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/minicluster-data/ts-1/wal/wals/9e21eb5c46c742c48adaba3408f00bac
I20260809 02:16:23.025971 27814 ts_tablet_manager.cc:1939] T 9e21eb5c46c742c48adaba3408f00bac P e33d0f78bb944d67ac12cec28e0bec5b: Deleting consensus metadata
I20260809 02:16:23.026007 27191 tablet_service.cc:1467] Processing CreateTablet for tablet fa074991209446d18a9208d8b510fd2f (DEFAULT_TABLE table=test-table [id=d8381d37afe441d7a968044f76f6922f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:16:23.026333 27191 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa074991209446d18a9208d8b510fd2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:16:23.026964 27816 ts_tablet_manager.cc:1918] T 9e21eb5c46c742c48adaba3408f00bac P a1a992abbf44400bbf100e73e082e99f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:16:23.027130 27816 log.cc:1196] T 9e21eb5c46c742c48adaba3408f00bac P a1a992abbf44400bbf100e73e082e99f: Deleting WAL directory at /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/minicluster-data/ts-0/wal/wals/9e21eb5c46c742c48adaba3408f00bac
I20260809 02:16:23.027458 27816 ts_tablet_manager.cc:1939] T 9e21eb5c46c742c48adaba3408f00bac P a1a992abbf44400bbf100e73e082e99f: Deleting consensus metadata
I20260809 02:16:23.028457 27777 tablet_bootstrap.cc:492] T fa074991209446d18a9208d8b510fd2f P a1a992abbf44400bbf100e73e082e99f: Bootstrap starting.
I20260809 02:16:23.028450 26955 catalog_manager.cc:4841] TS a1a992abbf44400bbf100e73e082e99f (127.15.75.129:41883): tablet 9e21eb5c46c742c48adaba3408f00bac (table test-table [id=788329e7f82c48e6ab1f220f142dc02e]) successfully deleted
I20260809 02:16:23.028787 26958 catalog_manager.cc:4841] TS e33d0f78bb944d67ac12cec28e0bec5b (127.15.75.130:44881): tablet 9e21eb5c46c742c48adaba3408f00bac (table test-table [id=788329e7f82c48e6ab1f220f142dc02e]) successfully deleted
I20260809 02:16:23.028939 27400 tablet_service.cc:1467] Processing CreateTablet for tablet fa074991209446d18a9208d8b510fd2f (DEFAULT_TABLE table=test-table [id=d8381d37afe441d7a968044f76f6922f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:16:23.029184 27400 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa074991209446d18a9208d8b510fd2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:16:23.029342 27626 tablet_service.cc:1467] Processing CreateTablet for tablet fa074991209446d18a9208d8b510fd2f (DEFAULT_TABLE table=test-table [id=d8381d37afe441d7a968044f76f6922f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:16:23.029527 27777 tablet_bootstrap.cc:654] T fa074991209446d18a9208d8b510fd2f P a1a992abbf44400bbf100e73e082e99f: Neither blocks nor log segments found. Creating new log.
I20260809 02:16:23.029585 27626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa074991209446d18a9208d8b510fd2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:16:23.030742 27777 tablet_bootstrap.cc:492] T fa074991209446d18a9208d8b510fd2f P a1a992abbf44400bbf100e73e082e99f: No bootstrap required, opened a new log
I20260809 02:16:23.030813 27777 ts_tablet_manager.cc:1397] T fa074991209446d18a9208d8b510fd2f P a1a992abbf44400bbf100e73e082e99f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 02:16:23.031273 27777 raft_consensus.cc:348] T fa074991209446d18a9208d8b510fd2f P a1a992abbf44400bbf100e73e082e99f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a992abbf44400bbf100e73e082e99f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 41883 } } peers { permanent_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 44881 } } peers { permanent_uuid: "621578d59c654ad6b768bf8ff3fd7209" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 38591 } }
I20260809 02:16:23.031378 27777 raft_consensus.cc:374] T fa074991209446d18a9208d8b510fd2f P a1a992abbf44400bbf100e73e082e99f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:16:23.031416 27777 raft_consensus.cc:729] T fa074991209446d18a9208d8b510fd2f P a1a992abbf44400bbf100e73e082e99f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1a992abbf44400bbf100e73e082e99f, State: Initialized, Role: FOLLOWER
I20260809 02:16:23.031531 27777 consensus_queue.cc:260] T fa074991209446d18a9208d8b510fd2f P a1a992abbf44400bbf100e73e082e99f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a992abbf44400bbf100e73e082e99f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 41883 } } peers { permanent_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 44881 } } peers { permanent_uuid: "621578d59c654ad6b768bf8ff3fd7209" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 38591 } }
I20260809 02:16:23.031808 27777 ts_tablet_manager.cc:1428] T fa074991209446d18a9208d8b510fd2f P a1a992abbf44400bbf100e73e082e99f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 02:16:23.032037 27775 tablet_bootstrap.cc:492] T fa074991209446d18a9208d8b510fd2f P e33d0f78bb944d67ac12cec28e0bec5b: Bootstrap starting.
I20260809 02:16:23.032745 27813 ts_tablet_manager.cc:1905] T 9e21eb5c46c742c48adaba3408f00bac P 621578d59c654ad6b768bf8ff3fd7209: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:16:23.032903 27780 tablet_bootstrap.cc:492] T fa074991209446d18a9208d8b510fd2f P 621578d59c654ad6b768bf8ff3fd7209: Bootstrap starting.
I20260809 02:16:23.033752 27780 tablet_bootstrap.cc:654] T fa074991209446d18a9208d8b510fd2f P 621578d59c654ad6b768bf8ff3fd7209: Neither blocks nor log segments found. Creating new log.
I20260809 02:16:23.035224 27813 ts_tablet_manager.cc:1918] T 9e21eb5c46c742c48adaba3408f00bac P 621578d59c654ad6b768bf8ff3fd7209: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:16:23.035327 27813 log.cc:1196] T 9e21eb5c46c742c48adaba3408f00bac P 621578d59c654ad6b768bf8ff3fd7209: Deleting WAL directory at /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/minicluster-data/ts-2/wal/wals/9e21eb5c46c742c48adaba3408f00bac
I20260809 02:16:23.035635 27813 ts_tablet_manager.cc:1939] T 9e21eb5c46c742c48adaba3408f00bac P 621578d59c654ad6b768bf8ff3fd7209: Deleting consensus metadata
I20260809 02:16:23.035982 27775 tablet_bootstrap.cc:654] T fa074991209446d18a9208d8b510fd2f P e33d0f78bb944d67ac12cec28e0bec5b: Neither blocks nor log segments found. Creating new log.
I20260809 02:16:23.036608 26959 catalog_manager.cc:4841] TS 621578d59c654ad6b768bf8ff3fd7209 (127.15.75.131:38591): tablet 9e21eb5c46c742c48adaba3408f00bac (table test-table [id=788329e7f82c48e6ab1f220f142dc02e]) successfully deleted
I20260809 02:16:23.037050 27780 tablet_bootstrap.cc:492] T fa074991209446d18a9208d8b510fd2f P 621578d59c654ad6b768bf8ff3fd7209: No bootstrap required, opened a new log
I20260809 02:16:23.037158 27780 ts_tablet_manager.cc:1397] T fa074991209446d18a9208d8b510fd2f P 621578d59c654ad6b768bf8ff3fd7209: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260809 02:16:23.037554 27780 raft_consensus.cc:348] T fa074991209446d18a9208d8b510fd2f P 621578d59c654ad6b768bf8ff3fd7209 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a992abbf44400bbf100e73e082e99f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 41883 } } peers { permanent_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 44881 } } peers { permanent_uuid: "621578d59c654ad6b768bf8ff3fd7209" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 38591 } }
I20260809 02:16:23.037644 27780 raft_consensus.cc:374] T fa074991209446d18a9208d8b510fd2f P 621578d59c654ad6b768bf8ff3fd7209 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:16:23.037665 27780 raft_consensus.cc:729] T fa074991209446d18a9208d8b510fd2f P 621578d59c654ad6b768bf8ff3fd7209 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 621578d59c654ad6b768bf8ff3fd7209, State: Initialized, Role: FOLLOWER
I20260809 02:16:23.037814 27780 consensus_queue.cc:260] T fa074991209446d18a9208d8b510fd2f P 621578d59c654ad6b768bf8ff3fd7209 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a992abbf44400bbf100e73e082e99f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 41883 } } peers { permanent_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 44881 } } peers { permanent_uuid: "621578d59c654ad6b768bf8ff3fd7209" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 38591 } }
I20260809 02:16:23.038076 27780 ts_tablet_manager.cc:1428] T fa074991209446d18a9208d8b510fd2f P 621578d59c654ad6b768bf8ff3fd7209: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 02:16:23.038785 27775 tablet_bootstrap.cc:492] T fa074991209446d18a9208d8b510fd2f P e33d0f78bb944d67ac12cec28e0bec5b: No bootstrap required, opened a new log
I20260809 02:16:23.038892 27775 ts_tablet_manager.cc:1397] T fa074991209446d18a9208d8b510fd2f P e33d0f78bb944d67ac12cec28e0bec5b: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260809 02:16:23.039311 27775 raft_consensus.cc:348] T fa074991209446d18a9208d8b510fd2f P e33d0f78bb944d67ac12cec28e0bec5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a992abbf44400bbf100e73e082e99f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 41883 } } peers { permanent_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 44881 } } peers { permanent_uuid: "621578d59c654ad6b768bf8ff3fd7209" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 38591 } }
I20260809 02:16:23.039422 27775 raft_consensus.cc:374] T fa074991209446d18a9208d8b510fd2f P e33d0f78bb944d67ac12cec28e0bec5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:16:23.039466 27775 raft_consensus.cc:729] T fa074991209446d18a9208d8b510fd2f P e33d0f78bb944d67ac12cec28e0bec5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e33d0f78bb944d67ac12cec28e0bec5b, State: Initialized, Role: FOLLOWER
I20260809 02:16:23.039594 27775 consensus_queue.cc:260] T fa074991209446d18a9208d8b510fd2f P e33d0f78bb944d67ac12cec28e0bec5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a992abbf44400bbf100e73e082e99f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 41883 } } peers { permanent_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 44881 } } peers { permanent_uuid: "621578d59c654ad6b768bf8ff3fd7209" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 38591 } }
I20260809 02:16:23.039897 27775 ts_tablet_manager.cc:1428] T fa074991209446d18a9208d8b510fd2f P e33d0f78bb944d67ac12cec28e0bec5b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20260809 02:16:23.050139 27290 tablet.cc:2369] T fa074991209446d18a9208d8b510fd2f P a1a992abbf44400bbf100e73e082e99f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:16:23.266566 27783 raft_consensus.cc:482] T fa074991209446d18a9208d8b510fd2f P e33d0f78bb944d67ac12cec28e0bec5b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:16:23.266727 27783 raft_consensus.cc:504] T fa074991209446d18a9208d8b510fd2f P e33d0f78bb944d67ac12cec28e0bec5b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a992abbf44400bbf100e73e082e99f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 41883 } } peers { permanent_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 44881 } } peers { permanent_uuid: "621578d59c654ad6b768bf8ff3fd7209" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 38591 } }
I20260809 02:16:23.268023 27783 leader_election.cc:288] T fa074991209446d18a9208d8b510fd2f P e33d0f78bb944d67ac12cec28e0bec5b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a1a992abbf44400bbf100e73e082e99f (127.15.75.129:41883), 621578d59c654ad6b768bf8ff3fd7209 (127.15.75.131:38591)
I20260809 02:16:23.273303 27771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:23.268463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.130:47053 (local address 127.15.75.131:38591)
0809 02:16:23.268596 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:23.268601 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:23.268834 (+ 233us) server_negotiation.cc:407] Connection header received
0809 02:16:23.269007 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:23.269013 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:23.269070 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:23.269111 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:16:23.269570 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:23.270361 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:23.272072 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:23.272670 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:23.272817 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:23.272873 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:23.273140 (+ 267us) server_negotiation.cc:299] Negotiation successful
0809 02:16:23.273205 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 02:16:23.273424 27824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:23.268195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.129:41883 (local address 127.15.75.130:55757)
0809 02:16:23.268658 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:23.268673 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:23.268863 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:23.269236 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:23.269240 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:23.269259 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:16:23.269446 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:23.269453 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:23.270486 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:23.270490 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:23.271939 (+ 1449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:23.271951 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:23.272829 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:23.272835 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:23.272975 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:23.273017 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 02:16:23.273101 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 02:16:23.273240 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373,"mutex_wait_us":171,"spinlock_wait_cycles":1280,"thread_start_us":88,"threads_started":1}
I20260809 02:16:23.273442 27822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:23.268334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.131:38591 (local address 127.15.75.130:47053)
0809 02:16:23.268759 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:23.268772 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:23.268884 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:23.269263 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:23.269265 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:23.269299 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:16:23.269446 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:23.269453 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:23.270486 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:23.270490 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:23.271943 (+ 1453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:23.271951 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:23.272844 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:23.272847 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:23.272971 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:23.273025 (+ 54us) client_negotiation.cc:769] Sending connection context
0809 02:16:23.273101 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 02:16:23.273237 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"mutex_wait_us":264,"spinlock_wait_cycles":26240,"thread_start_us":77,"threads_started":1}
I20260809 02:16:23.273777 27772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:23.268514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.130:55757 (local address 127.15.75.129:41883)
0809 02:16:23.268640 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:23.268644 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:23.268770 (+ 126us) server_negotiation.cc:407] Connection header received
0809 02:16:23.269042 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:23.269045 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:23.269109 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:23.269175 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:16:23.269570 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:23.270359 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:23.272072 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:23.272670 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:23.272817 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:23.272898 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:23.273542 (+ 644us) server_negotiation.cc:299] Negotiation successful
0809 02:16:23.273646 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 02:16:23.273931 27656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa074991209446d18a9208d8b510fd2f" candidate_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "621578d59c654ad6b768bf8ff3fd7209" is_pre_election: true
I20260809 02:16:23.274130 27656 raft_consensus.cc:2454] T fa074991209446d18a9208d8b510fd2f P 621578d59c654ad6b768bf8ff3fd7209 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e33d0f78bb944d67ac12cec28e0bec5b in term 0.
I20260809 02:16:23.274345 27213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa074991209446d18a9208d8b510fd2f" candidate_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1a992abbf44400bbf100e73e082e99f" is_pre_election: true
I20260809 02:16:23.274453 27213 raft_consensus.cc:2454] T fa074991209446d18a9208d8b510fd2f P a1a992abbf44400bbf100e73e082e99f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e33d0f78bb944d67ac12cec28e0bec5b in term 0.
I20260809 02:16:23.274569 27335 leader_election.cc:302] T fa074991209446d18a9208d8b510fd2f P e33d0f78bb944d67ac12cec28e0bec5b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 621578d59c654ad6b768bf8ff3fd7209, e33d0f78bb944d67ac12cec28e0bec5b; no voters:
I20260809 02:16:23.274869 27783 raft_consensus.cc:2781] T fa074991209446d18a9208d8b510fd2f P e33d0f78bb944d67ac12cec28e0bec5b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:16:23.274927 27783 raft_consensus.cc:482] T fa074991209446d18a9208d8b510fd2f P e33d0f78bb944d67ac12cec28e0bec5b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:16:23.274955 27783 raft_consensus.cc:3037] T fa074991209446d18a9208d8b510fd2f P e33d0f78bb944d67ac12cec28e0bec5b [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:16:23.275991 27783 raft_consensus.cc:504] T fa074991209446d18a9208d8b510fd2f P e33d0f78bb944d67ac12cec28e0bec5b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a992abbf44400bbf100e73e082e99f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 41883 } } peers { permanent_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 44881 } } peers { permanent_uuid: "621578d59c654ad6b768bf8ff3fd7209" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 38591 } }
I20260809 02:16:23.276335 27783 leader_election.cc:288] T fa074991209446d18a9208d8b510fd2f P e33d0f78bb944d67ac12cec28e0bec5b [CANDIDATE]: Term 1 election: Requested vote from peers a1a992abbf44400bbf100e73e082e99f (127.15.75.129:41883), 621578d59c654ad6b768bf8ff3fd7209 (127.15.75.131:38591)
I20260809 02:16:23.276741 27656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa074991209446d18a9208d8b510fd2f" candidate_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "621578d59c654ad6b768bf8ff3fd7209"
I20260809 02:16:23.276741 27213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa074991209446d18a9208d8b510fd2f" candidate_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1a992abbf44400bbf100e73e082e99f"
I20260809 02:16:23.276870 27213 raft_consensus.cc:3037] T fa074991209446d18a9208d8b510fd2f P a1a992abbf44400bbf100e73e082e99f [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:16:23.276870 27656 raft_consensus.cc:3037] T fa074991209446d18a9208d8b510fd2f P 621578d59c654ad6b768bf8ff3fd7209 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:16:23.278045 27213 raft_consensus.cc:2454] T fa074991209446d18a9208d8b510fd2f P a1a992abbf44400bbf100e73e082e99f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e33d0f78bb944d67ac12cec28e0bec5b in term 1.
I20260809 02:16:23.278075 27656 raft_consensus.cc:2454] T fa074991209446d18a9208d8b510fd2f P 621578d59c654ad6b768bf8ff3fd7209 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e33d0f78bb944d67ac12cec28e0bec5b in term 1.
I20260809 02:16:23.278339 27335 leader_election.cc:302] T fa074991209446d18a9208d8b510fd2f P e33d0f78bb944d67ac12cec28e0bec5b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 621578d59c654ad6b768bf8ff3fd7209, e33d0f78bb944d67ac12cec28e0bec5b; no voters:
I20260809 02:16:23.278514 27783 raft_consensus.cc:2781] T fa074991209446d18a9208d8b510fd2f P e33d0f78bb944d67ac12cec28e0bec5b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:16:23.278712 27783 raft_consensus.cc:686] T fa074991209446d18a9208d8b510fd2f P e33d0f78bb944d67ac12cec28e0bec5b [term 1 LEADER]: Becoming Leader. State: Replica: e33d0f78bb944d67ac12cec28e0bec5b, State: Running, Role: LEADER
I20260809 02:16:23.279000 27783 consensus_queue.cc:237] T fa074991209446d18a9208d8b510fd2f P e33d0f78bb944d67ac12cec28e0bec5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a992abbf44400bbf100e73e082e99f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 41883 } } peers { permanent_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 44881 } } peers { permanent_uuid: "621578d59c654ad6b768bf8ff3fd7209" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 38591 } }
W20260809 02:16:23.280269 27504 tablet.cc:2369] T fa074991209446d18a9208d8b510fd2f P e33d0f78bb944d67ac12cec28e0bec5b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:16:23.281358 26975 catalog_manager.cc:5495] T fa074991209446d18a9208d8b510fd2f P e33d0f78bb944d67ac12cec28e0bec5b reported cstate change: term changed from 0 to 1, leader changed from <none> to e33d0f78bb944d67ac12cec28e0bec5b (127.15.75.130). New cstate: current_term: 1 leader_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a992abbf44400bbf100e73e082e99f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 41883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 44881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "621578d59c654ad6b768bf8ff3fd7209" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 38591 } health_report { overall_health: UNKNOWN } } }
Aug 09 02:16:23 dist-test-slave-s11t krb5kdc[26909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241781, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.75.190@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM
I20260809 02:16:23.354983 27836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:23.346330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:38449 (local address 127.0.0.1:48566)
0809 02:16:23.346645 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:23.346660 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:23.346818 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:23.347077 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:23.347080 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:23.347565 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:23.347660 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:23.347667 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:23.348441 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:23.348448 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:23.349083 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:23.349093 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:23.349547 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:23.349550 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:23.349623 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:23.350239 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:23.350251 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:23.351669 (+ 1418us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:16:23.353381 (+ 1712us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:23.353393 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:23.353400 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:23.353628 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:23.353907 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:23.353912 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:23.353914 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:23.353997 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:23.354380 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:16:23.354388 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:16:23.354501 (+ 113us) client_negotiation.cc:769] Sending connection context
0809 02:16:23.354614 (+ 113us) client_negotiation.cc:240] Negotiation successful
0809 02:16:23.354779 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":87,"threads_started":1}
I20260809 02:16:23.354983 27837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:23.346387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48566 (local address 127.15.75.190:38449)
0809 02:16:23.346624 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:23.346635 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:23.346743 (+ 108us) server_negotiation.cc:407] Connection header received
0809 02:16:23.346924 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:23.346926 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:23.346971 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:23.347059 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:23.347784 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:23.348333 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:23.349204 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:23.349434 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:23.349547 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:23.351807 (+ 2260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:23.351812 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:23.351815 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:23.351844 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:23.353265 (+ 1421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:23.353730 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:23.353735 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:23.353737 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:23.353817 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:23.354093 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:23.354096 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:23.354097 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:23.354275 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:23.354380 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:23.354672 (+ 292us) server_negotiation.cc:299] Negotiation successful
0809 02:16:23.354829 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":171,"thread_start_us":95,"threads_started":1}
I20260809 02:16:23.359062 26975 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.15.75.190@KRBTEST.COM'} at 127.0.0.1:48566:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 02:16:23.359772 26975 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:16:23.364390 27400 tablet_service.cc:1467] Processing CreateTablet for tablet 4e89dbffcdda45c8935f6a729feeca08 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e2a7fd96d95d4abcb87570dc5bb422b9]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 02:16:23.364390 27191 tablet_service.cc:1467] Processing CreateTablet for tablet 4e89dbffcdda45c8935f6a729feeca08 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e2a7fd96d95d4abcb87570dc5bb422b9]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 02:16:23.364526 27626 tablet_service.cc:1467] Processing CreateTablet for tablet 4e89dbffcdda45c8935f6a729feeca08 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e2a7fd96d95d4abcb87570dc5bb422b9]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 02:16:23.364818 27400 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e89dbffcdda45c8935f6a729feeca08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:16:23.364818 27191 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e89dbffcdda45c8935f6a729feeca08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:16:23.365008 27626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e89dbffcdda45c8935f6a729feeca08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:16:23.367434 27780 tablet_bootstrap.cc:492] T 4e89dbffcdda45c8935f6a729feeca08 P 621578d59c654ad6b768bf8ff3fd7209: Bootstrap starting.
I20260809 02:16:23.367687 27777 tablet_bootstrap.cc:492] T 4e89dbffcdda45c8935f6a729feeca08 P a1a992abbf44400bbf100e73e082e99f: Bootstrap starting.
I20260809 02:16:23.367687 27775 tablet_bootstrap.cc:492] T 4e89dbffcdda45c8935f6a729feeca08 P e33d0f78bb944d67ac12cec28e0bec5b: Bootstrap starting.
I20260809 02:16:23.368371 27780 tablet_bootstrap.cc:654] T 4e89dbffcdda45c8935f6a729feeca08 P 621578d59c654ad6b768bf8ff3fd7209: Neither blocks nor log segments found. Creating new log.
I20260809 02:16:23.368558 27775 tablet_bootstrap.cc:654] T 4e89dbffcdda45c8935f6a729feeca08 P e33d0f78bb944d67ac12cec28e0bec5b: Neither blocks nor log segments found. Creating new log.
I20260809 02:16:23.368559 27777 tablet_bootstrap.cc:654] T 4e89dbffcdda45c8935f6a729feeca08 P a1a992abbf44400bbf100e73e082e99f: Neither blocks nor log segments found. Creating new log.
I20260809 02:16:23.369643 27780 tablet_bootstrap.cc:492] T 4e89dbffcdda45c8935f6a729feeca08 P 621578d59c654ad6b768bf8ff3fd7209: No bootstrap required, opened a new log
I20260809 02:16:23.369753 27780 ts_tablet_manager.cc:1397] T 4e89dbffcdda45c8935f6a729feeca08 P 621578d59c654ad6b768bf8ff3fd7209: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260809 02:16:23.369958 27777 tablet_bootstrap.cc:492] T 4e89dbffcdda45c8935f6a729feeca08 P a1a992abbf44400bbf100e73e082e99f: No bootstrap required, opened a new log
I20260809 02:16:23.370030 27777 ts_tablet_manager.cc:1397] T 4e89dbffcdda45c8935f6a729feeca08 P a1a992abbf44400bbf100e73e082e99f: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260809 02:16:23.370172 27780 raft_consensus.cc:348] T 4e89dbffcdda45c8935f6a729feeca08 P 621578d59c654ad6b768bf8ff3fd7209 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a992abbf44400bbf100e73e082e99f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 41883 } } peers { permanent_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 44881 } } peers { permanent_uuid: "621578d59c654ad6b768bf8ff3fd7209" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 38591 } }
I20260809 02:16:23.370296 27780 raft_consensus.cc:374] T 4e89dbffcdda45c8935f6a729feeca08 P 621578d59c654ad6b768bf8ff3fd7209 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:16:23.370342 27780 raft_consensus.cc:729] T 4e89dbffcdda45c8935f6a729feeca08 P 621578d59c654ad6b768bf8ff3fd7209 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 621578d59c654ad6b768bf8ff3fd7209, State: Initialized, Role: FOLLOWER
I20260809 02:16:23.370474 27780 consensus_queue.cc:260] T 4e89dbffcdda45c8935f6a729feeca08 P 621578d59c654ad6b768bf8ff3fd7209 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a992abbf44400bbf100e73e082e99f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 41883 } } peers { permanent_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 44881 } } peers { permanent_uuid: "621578d59c654ad6b768bf8ff3fd7209" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 38591 } }
I20260809 02:16:23.370477 27777 raft_consensus.cc:348] T 4e89dbffcdda45c8935f6a729feeca08 P a1a992abbf44400bbf100e73e082e99f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a992abbf44400bbf100e73e082e99f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 41883 } } peers { permanent_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 44881 } } peers { permanent_uuid: "621578d59c654ad6b768bf8ff3fd7209" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 38591 } }
I20260809 02:16:23.370577 27777 raft_consensus.cc:374] T 4e89dbffcdda45c8935f6a729feeca08 P a1a992abbf44400bbf100e73e082e99f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:16:23.370627 27777 raft_consensus.cc:729] T 4e89dbffcdda45c8935f6a729feeca08 P a1a992abbf44400bbf100e73e082e99f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1a992abbf44400bbf100e73e082e99f, State: Initialized, Role: FOLLOWER
I20260809 02:16:23.370772 27780 ts_tablet_manager.cc:1428] T 4e89dbffcdda45c8935f6a729feeca08 P 621578d59c654ad6b768bf8ff3fd7209: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260809 02:16:23.370765 27777 consensus_queue.cc:260] T 4e89dbffcdda45c8935f6a729feeca08 P a1a992abbf44400bbf100e73e082e99f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a992abbf44400bbf100e73e082e99f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 41883 } } peers { permanent_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 44881 } } peers { permanent_uuid: "621578d59c654ad6b768bf8ff3fd7209" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 38591 } }
I20260809 02:16:23.370999 27777 ts_tablet_manager.cc:1428] T 4e89dbffcdda45c8935f6a729feeca08 P a1a992abbf44400bbf100e73e082e99f: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260809 02:16:23.370980 27808 tablet_replica.cc:439] T 4e89dbffcdda45c8935f6a729feeca08 P 621578d59c654ad6b768bf8ff3fd7209: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a992abbf44400bbf100e73e082e99f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 41883 } } peers { permanent_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 44881 } } peers { permanent_uuid: "621578d59c654ad6b768bf8ff3fd7209" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 38591 } } }
I20260809 02:16:23.371097 27808 tablet_replica.cc:442] T 4e89dbffcdda45c8935f6a729feeca08 P 621578d59c654ad6b768bf8ff3fd7209: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 02:16:23.371541 27775 tablet_bootstrap.cc:492] T 4e89dbffcdda45c8935f6a729feeca08 P e33d0f78bb944d67ac12cec28e0bec5b: No bootstrap required, opened a new log
I20260809 02:16:23.371636 27775 ts_tablet_manager.cc:1397] T 4e89dbffcdda45c8935f6a729feeca08 P e33d0f78bb944d67ac12cec28e0bec5b: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260809 02:16:23.372105 27775 raft_consensus.cc:348] T 4e89dbffcdda45c8935f6a729feeca08 P e33d0f78bb944d67ac12cec28e0bec5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a992abbf44400bbf100e73e082e99f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 41883 } } peers { permanent_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 44881 } } peers { permanent_uuid: "621578d59c654ad6b768bf8ff3fd7209" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 38591 } }
I20260809 02:16:23.372227 27775 raft_consensus.cc:374] T 4e89dbffcdda45c8935f6a729feeca08 P e33d0f78bb944d67ac12cec28e0bec5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:16:23.372293 27775 raft_consensus.cc:729] T 4e89dbffcdda45c8935f6a729feeca08 P e33d0f78bb944d67ac12cec28e0bec5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e33d0f78bb944d67ac12cec28e0bec5b, State: Initialized, Role: FOLLOWER
I20260809 02:16:23.372426 27775 consensus_queue.cc:260] T 4e89dbffcdda45c8935f6a729feeca08 P e33d0f78bb944d67ac12cec28e0bec5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a992abbf44400bbf100e73e082e99f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 41883 } } peers { permanent_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 44881 } } peers { permanent_uuid: "621578d59c654ad6b768bf8ff3fd7209" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 38591 } }
I20260809 02:16:23.372658 27783 tablet_replica.cc:439] T 4e89dbffcdda45c8935f6a729feeca08 P e33d0f78bb944d67ac12cec28e0bec5b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a992abbf44400bbf100e73e082e99f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 41883 } } peers { permanent_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 44881 } } peers { permanent_uuid: "621578d59c654ad6b768bf8ff3fd7209" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 38591 } } }
I20260809 02:16:23.372818 27783 tablet_replica.cc:442] T 4e89dbffcdda45c8935f6a729feeca08 P e33d0f78bb944d67ac12cec28e0bec5b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 02:16:23.373210 27785 tablet_replica.cc:439] T 4e89dbffcdda45c8935f6a729feeca08 P a1a992abbf44400bbf100e73e082e99f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a992abbf44400bbf100e73e082e99f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 41883 } } peers { permanent_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 44881 } } peers { permanent_uuid: "621578d59c654ad6b768bf8ff3fd7209" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 38591 } } }
I20260809 02:16:23.373345 27785 tablet_replica.cc:442] T 4e89dbffcdda45c8935f6a729feeca08 P a1a992abbf44400bbf100e73e082e99f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 02:16:23.372752 27775 ts_tablet_manager.cc:1428] T 4e89dbffcdda45c8935f6a729feeca08 P e33d0f78bb944d67ac12cec28e0bec5b: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260809 02:16:23.457360 27785 raft_consensus.cc:482] T 4e89dbffcdda45c8935f6a729feeca08 P a1a992abbf44400bbf100e73e082e99f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:16:23.457502 27785 raft_consensus.cc:504] T 4e89dbffcdda45c8935f6a729feeca08 P a1a992abbf44400bbf100e73e082e99f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a992abbf44400bbf100e73e082e99f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 41883 } } peers { permanent_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 44881 } } peers { permanent_uuid: "621578d59c654ad6b768bf8ff3fd7209" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 38591 } }
I20260809 02:16:23.458065 27785 leader_election.cc:288] T 4e89dbffcdda45c8935f6a729feeca08 P a1a992abbf44400bbf100e73e082e99f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e33d0f78bb944d67ac12cec28e0bec5b (127.15.75.130:44881), 621578d59c654ad6b768bf8ff3fd7209 (127.15.75.131:38591)
I20260809 02:16:23.458549 27656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e89dbffcdda45c8935f6a729feeca08" candidate_uuid: "a1a992abbf44400bbf100e73e082e99f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "621578d59c654ad6b768bf8ff3fd7209" is_pre_election: true
I20260809 02:16:23.458549 27430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e89dbffcdda45c8935f6a729feeca08" candidate_uuid: "a1a992abbf44400bbf100e73e082e99f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" is_pre_election: true
I20260809 02:16:23.458725 27656 raft_consensus.cc:2454] T 4e89dbffcdda45c8935f6a729feeca08 P 621578d59c654ad6b768bf8ff3fd7209 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1a992abbf44400bbf100e73e082e99f in term 0.
I20260809 02:16:23.458725 27430 raft_consensus.cc:2454] T 4e89dbffcdda45c8935f6a729feeca08 P e33d0f78bb944d67ac12cec28e0bec5b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1a992abbf44400bbf100e73e082e99f in term 0.
I20260809 02:16:23.459111 27120 leader_election.cc:302] T 4e89dbffcdda45c8935f6a729feeca08 P a1a992abbf44400bbf100e73e082e99f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 621578d59c654ad6b768bf8ff3fd7209, a1a992abbf44400bbf100e73e082e99f, e33d0f78bb944d67ac12cec28e0bec5b; no voters:
I20260809 02:16:23.459349 27785 raft_consensus.cc:2781] T 4e89dbffcdda45c8935f6a729feeca08 P a1a992abbf44400bbf100e73e082e99f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:16:23.459426 27785 raft_consensus.cc:482] T 4e89dbffcdda45c8935f6a729feeca08 P a1a992abbf44400bbf100e73e082e99f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:16:23.459479 27785 raft_consensus.cc:3037] T 4e89dbffcdda45c8935f6a729feeca08 P a1a992abbf44400bbf100e73e082e99f [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:16:23.460496 27785 raft_consensus.cc:504] T 4e89dbffcdda45c8935f6a729feeca08 P a1a992abbf44400bbf100e73e082e99f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a992abbf44400bbf100e73e082e99f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 41883 } } peers { permanent_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 44881 } } peers { permanent_uuid: "621578d59c654ad6b768bf8ff3fd7209" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 38591 } }
I20260809 02:16:23.460865 27785 leader_election.cc:288] T 4e89dbffcdda45c8935f6a729feeca08 P a1a992abbf44400bbf100e73e082e99f [CANDIDATE]: Term 1 election: Requested vote from peers e33d0f78bb944d67ac12cec28e0bec5b (127.15.75.130:44881), 621578d59c654ad6b768bf8ff3fd7209 (127.15.75.131:38591)
I20260809 02:16:23.461238 27430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e89dbffcdda45c8935f6a729feeca08" candidate_uuid: "a1a992abbf44400bbf100e73e082e99f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e33d0f78bb944d67ac12cec28e0bec5b"
I20260809 02:16:23.461247 27656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e89dbffcdda45c8935f6a729feeca08" candidate_uuid: "a1a992abbf44400bbf100e73e082e99f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "621578d59c654ad6b768bf8ff3fd7209"
I20260809 02:16:23.461385 27430 raft_consensus.cc:3037] T 4e89dbffcdda45c8935f6a729feeca08 P e33d0f78bb944d67ac12cec28e0bec5b [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:16:23.461397 27656 raft_consensus.cc:3037] T 4e89dbffcdda45c8935f6a729feeca08 P 621578d59c654ad6b768bf8ff3fd7209 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:16:23.462322 27430 raft_consensus.cc:2454] T 4e89dbffcdda45c8935f6a729feeca08 P e33d0f78bb944d67ac12cec28e0bec5b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1a992abbf44400bbf100e73e082e99f in term 1.
I20260809 02:16:23.462518 27656 raft_consensus.cc:2454] T 4e89dbffcdda45c8935f6a729feeca08 P 621578d59c654ad6b768bf8ff3fd7209 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1a992abbf44400bbf100e73e082e99f in term 1.
I20260809 02:16:23.462641 27120 leader_election.cc:302] T 4e89dbffcdda45c8935f6a729feeca08 P a1a992abbf44400bbf100e73e082e99f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a1a992abbf44400bbf100e73e082e99f, e33d0f78bb944d67ac12cec28e0bec5b; no voters:
I20260809 02:16:23.462841 27785 raft_consensus.cc:2781] T 4e89dbffcdda45c8935f6a729feeca08 P a1a992abbf44400bbf100e73e082e99f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:16:23.463039 27785 raft_consensus.cc:686] T 4e89dbffcdda45c8935f6a729feeca08 P a1a992abbf44400bbf100e73e082e99f [term 1 LEADER]: Becoming Leader. State: Replica: a1a992abbf44400bbf100e73e082e99f, State: Running, Role: LEADER
I20260809 02:16:23.463326 27785 consensus_queue.cc:237] T 4e89dbffcdda45c8935f6a729feeca08 P a1a992abbf44400bbf100e73e082e99f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a992abbf44400bbf100e73e082e99f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 41883 } } peers { permanent_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 44881 } } peers { permanent_uuid: "621578d59c654ad6b768bf8ff3fd7209" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 38591 } }
I20260809 02:16:23.464654 27844 tablet_replica.cc:439] T 4e89dbffcdda45c8935f6a729feeca08 P a1a992abbf44400bbf100e73e082e99f: TxnStatusTablet state changed. Reason: New leader a1a992abbf44400bbf100e73e082e99f. Latest consensus state: current_term: 1 leader_uuid: "a1a992abbf44400bbf100e73e082e99f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a992abbf44400bbf100e73e082e99f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 41883 } } peers { permanent_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 44881 } } peers { permanent_uuid: "621578d59c654ad6b768bf8ff3fd7209" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 38591 } } }
I20260809 02:16:23.464833 27844 tablet_replica.cc:442] T 4e89dbffcdda45c8935f6a729feeca08 P a1a992abbf44400bbf100e73e082e99f: This TxnStatusTablet replica's current role is: LEADER
I20260809 02:16:23.465128 27849 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 02:16:23.465304 27849 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 02:16:23.466085 26977 catalog_manager.cc:5495] T 4e89dbffcdda45c8935f6a729feeca08 P a1a992abbf44400bbf100e73e082e99f reported cstate change: term changed from 0 to 1, leader changed from <none> to a1a992abbf44400bbf100e73e082e99f (127.15.75.129). New cstate: current_term: 1 leader_uuid: "a1a992abbf44400bbf100e73e082e99f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a992abbf44400bbf100e73e082e99f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 41883 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 44881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "621578d59c654ad6b768bf8ff3fd7209" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 38591 } health_report { overall_health: UNKNOWN } } }
I20260809 02:16:23.499318 27772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:23.495440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40756 (local address 127.15.75.129:41883)
0809 02:16:23.495581 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:23.495586 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:23.495601 (+ 15us) server_negotiation.cc:407] Connection header received
0809 02:16:23.495693 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:23.495696 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:23.495759 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:23.495863 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:16:23.496110 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:23.496795 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:23.497793 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:23.498044 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:23.498154 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:23.498487 (+ 333us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:16:23.498888 (+ 401us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:16:23.499021 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:23.499106 (+ 85us) server_negotiation.cc:299] Negotiation successful
0809 02:16:23.499157 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 02:16:23.499295 27836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:23.495211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.129:41883 (local address 127.0.0.1:40756)
0809 02:16:23.495423 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:23.495438 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:23.495564 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:23.495877 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:23.495880 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:23.495888 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 02:16:23.496000 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:23.496006 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:23.496904 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:23.496910 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:23.497667 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:23.497678 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:23.498156 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:23.498160 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:23.498260 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:23.498302 (+ 42us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 02:16:23.498999 (+ 697us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 02:16:23.499017 (+ 18us) client_negotiation.cc:769] Sending connection context
0809 02:16:23.499086 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 02:16:23.499174 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74}
I20260809 02:16:23.502658 27430 raft_consensus.cc:1261] T 4e89dbffcdda45c8935f6a729feeca08 P e33d0f78bb944d67ac12cec28e0bec5b [term 1 FOLLOWER]: Refusing update from remote peer a1a992abbf44400bbf100e73e082e99f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:16:23.502751 27656 raft_consensus.cc:1261] T 4e89dbffcdda45c8935f6a729feeca08 P 621578d59c654ad6b768bf8ff3fd7209 [term 1 FOLLOWER]: Refusing update from remote peer a1a992abbf44400bbf100e73e082e99f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:16:23.503204 27785 consensus_queue.cc:1035] T 4e89dbffcdda45c8935f6a729feeca08 P a1a992abbf44400bbf100e73e082e99f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 44881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:16:23.503369 27844 consensus_queue.cc:1035] T 4e89dbffcdda45c8935f6a729feeca08 P a1a992abbf44400bbf100e73e082e99f [LEADER]: Connected to new peer: Peer: permanent_uuid: "621578d59c654ad6b768bf8ff3fd7209" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 38591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:16:23.504809 27783 tablet_replica.cc:439] T 4e89dbffcdda45c8935f6a729feeca08 P e33d0f78bb944d67ac12cec28e0bec5b: TxnStatusTablet state changed. Reason: New leader a1a992abbf44400bbf100e73e082e99f. Latest consensus state: current_term: 1 leader_uuid: "a1a992abbf44400bbf100e73e082e99f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a992abbf44400bbf100e73e082e99f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 41883 } } peers { permanent_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 44881 } } peers { permanent_uuid: "621578d59c654ad6b768bf8ff3fd7209" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 38591 } } }
I20260809 02:16:23.504964 27783 tablet_replica.cc:442] T 4e89dbffcdda45c8935f6a729feeca08 P e33d0f78bb944d67ac12cec28e0bec5b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 02:16:23.506446 27808 tablet_replica.cc:439] T 4e89dbffcdda45c8935f6a729feeca08 P 621578d59c654ad6b768bf8ff3fd7209: TxnStatusTablet state changed. Reason: New leader a1a992abbf44400bbf100e73e082e99f. Latest consensus state: current_term: 1 leader_uuid: "a1a992abbf44400bbf100e73e082e99f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a992abbf44400bbf100e73e082e99f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 41883 } } peers { permanent_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 44881 } } peers { permanent_uuid: "621578d59c654ad6b768bf8ff3fd7209" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 38591 } } }
I20260809 02:16:23.506577 27808 tablet_replica.cc:442] T 4e89dbffcdda45c8935f6a729feeca08 P 621578d59c654ad6b768bf8ff3fd7209: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 02:16:23.507719 27785 tablet_replica.cc:439] T 4e89dbffcdda45c8935f6a729feeca08 P a1a992abbf44400bbf100e73e082e99f: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a1a992abbf44400bbf100e73e082e99f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a992abbf44400bbf100e73e082e99f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 41883 } } peers { permanent_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 44881 } } peers { permanent_uuid: "621578d59c654ad6b768bf8ff3fd7209" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 38591 } } }
I20260809 02:16:23.507849 27785 tablet_replica.cc:442] T 4e89dbffcdda45c8935f6a729feeca08 P a1a992abbf44400bbf100e73e082e99f: This TxnStatusTablet replica's current role is: LEADER
I20260809 02:16:23.508160 27785 tablet_replica.cc:439] T 4e89dbffcdda45c8935f6a729feeca08 P a1a992abbf44400bbf100e73e082e99f: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a1a992abbf44400bbf100e73e082e99f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a992abbf44400bbf100e73e082e99f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 41883 } } peers { permanent_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 44881 } } peers { permanent_uuid: "621578d59c654ad6b768bf8ff3fd7209" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 38591 } } }
I20260809 02:16:23.508263 27785 tablet_replica.cc:442] T 4e89dbffcdda45c8935f6a729feeca08 P a1a992abbf44400bbf100e73e082e99f: This TxnStatusTablet replica's current role is: LEADER
I20260809 02:16:23.509786 27783 tablet_replica.cc:439] T 4e89dbffcdda45c8935f6a729feeca08 P e33d0f78bb944d67ac12cec28e0bec5b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a1a992abbf44400bbf100e73e082e99f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a992abbf44400bbf100e73e082e99f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 41883 } } peers { permanent_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 44881 } } peers { permanent_uuid: "621578d59c654ad6b768bf8ff3fd7209" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 38591 } } }
I20260809 02:16:23.509946 27783 tablet_replica.cc:442] T 4e89dbffcdda45c8935f6a729feeca08 P e33d0f78bb944d67ac12cec28e0bec5b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 02:16:23.510489 27808 tablet_replica.cc:439] T 4e89dbffcdda45c8935f6a729feeca08 P 621578d59c654ad6b768bf8ff3fd7209: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a1a992abbf44400bbf100e73e082e99f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a992abbf44400bbf100e73e082e99f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 41883 } } peers { permanent_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 44881 } } peers { permanent_uuid: "621578d59c654ad6b768bf8ff3fd7209" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 38591 } } }
I20260809 02:16:23.510602 27808 tablet_replica.cc:442] T 4e89dbffcdda45c8935f6a729feeca08 P 621578d59c654ad6b768bf8ff3fd7209: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 02:16:23.511096 27856 mvcc.cc:205] Tried to move back new op lower bound from 7316446345221181440 to 7316446345068519424. Current Snapshot: MvccSnapshot[applied={T|T < 7316446345221181440 or (T in {7316446345221181440})}]
I20260809 02:16:23.511294 27858 mvcc.cc:205] Tried to move back new op lower bound from 7316446345221181440 to 7316446345068519424. Current Snapshot: MvccSnapshot[applied={T|T < 7316446345221181440}]
I20260809 02:16:23.517755 27870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:23.513442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50814 (local address 127.15.75.130:44881)
0809 02:16:23.513684 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:23.513696 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:23.513742 (+ 46us) server_negotiation.cc:407] Connection header received
0809 02:16:23.513926 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:23.513928 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:23.513973 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:23.514067 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:16:23.514351 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:23.514867 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:23.516185 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:23.516421 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:23.516498 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:23.516846 (+ 348us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:16:23.517295 (+ 449us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:16:23.517384 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:23.517501 (+ 117us) server_negotiation.cc:299] Negotiation successful
0809 02:16:23.517548 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":69,"threads_started":1}
I20260809 02:16:23.529465 27772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:23.525105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.130:60379 (local address 127.15.75.129:41883)
0809 02:16:23.525257 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:23.525262 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:23.525384 (+ 122us) server_negotiation.cc:407] Connection header received
0809 02:16:23.525540 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:23.525545 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:23.525608 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:23.525695 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:16:23.525993 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:23.526498 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:23.528204 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:23.528474 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:23.528572 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:23.529038 (+ 466us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:16:23.529116 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:16:23.529194 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:23.529288 (+ 94us) server_negotiation.cc:299] Negotiation successful
0809 02:16:23.529335 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 02:16:23.529513 27874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:23.524984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.129:41883 (local address 127.15.75.130:60379)
0809 02:16:23.525296 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:23.525314 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:23.525437 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:23.525718 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:23.525721 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:23.525745 (+ 24us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 02:16:23.525872 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:23.525879 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:23.526607 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:23.526612 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:23.528083 (+ 1471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:23.528093 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:23.528654 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:23.528657 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:23.528854 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:23.528900 (+ 46us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 02:16:23.529198 (+ 298us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 02:16:23.529206 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 02:16:23.529268 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 02:16:23.529345 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":109,"threads_started":1}
I20260809 02:16:23.539891 27656 raft_consensus.cc:1261] T fa074991209446d18a9208d8b510fd2f P 621578d59c654ad6b768bf8ff3fd7209 [term 1 FOLLOWER]: Refusing update from remote peer e33d0f78bb944d67ac12cec28e0bec5b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:16:23.540277 27213 raft_consensus.cc:1261] T fa074991209446d18a9208d8b510fd2f P a1a992abbf44400bbf100e73e082e99f [term 1 FOLLOWER]: Refusing update from remote peer e33d0f78bb944d67ac12cec28e0bec5b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:16:23.540776 27825 consensus_queue.cc:1035] T fa074991209446d18a9208d8b510fd2f P e33d0f78bb944d67ac12cec28e0bec5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1a992abbf44400bbf100e73e082e99f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 41883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:16:23.541210 27825 consensus_queue.cc:1035] T fa074991209446d18a9208d8b510fd2f P e33d0f78bb944d67ac12cec28e0bec5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "621578d59c654ad6b768bf8ff3fd7209" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 38591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:16:23.565825 27870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:23.561416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.129:35299 (local address 127.15.75.130:44881)
0809 02:16:23.561653 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:23.561658 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:23.561759 (+ 101us) server_negotiation.cc:407] Connection header received
0809 02:16:23.562330 (+ 571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:23.562333 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:23.562378 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:23.562462 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:16:23.562778 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:23.563511 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:23.564584 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:23.564901 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:23.565018 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:23.565340 (+ 322us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:16:23.565421 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:16:23.565513 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:23.565647 (+ 134us) server_negotiation.cc:299] Negotiation successful
0809 02:16:23.565706 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163}
I20260809 02:16:23.565909 27887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:23.561363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.130:44881 (local address 127.15.75.129:35299)
0809 02:16:23.561686 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:23.561701 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:23.562188 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:23.562482 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:23.562486 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:23.562495 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 02:16:23.562643 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:23.562650 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:23.563627 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:23.563632 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:23.564463 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:23.564473 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:23.565031 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:23.565035 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:23.565146 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:23.565190 (+ 44us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 02:16:23.565541 (+ 351us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 02:16:23.565549 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 02:16:23.565633 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 02:16:23.565723 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":107,"threads_started":1}
I20260809 02:16:23.590735 26977 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48562:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:16:23.590909 26977 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48562:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:16:23.592890 26977 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 34bc02e2aabc462baa95d3fc45949cba: Sending DeleteTablet for 3 replicas of tablet fa074991209446d18a9208d8b510fd2f
I20260809 02:16:23.593377 27191 tablet_service.cc:1514] Processing DeleteTablet for tablet fa074991209446d18a9208d8b510fd2f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:16:23 UTC) from {username='oryx', principal='oryx/127.15.75.190@KRBTEST.COM'} at 127.0.0.1:40740
I20260809 02:16:23.593575 27400 tablet_service.cc:1514] Processing DeleteTablet for tablet fa074991209446d18a9208d8b510fd2f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:16:23 UTC) from {username='oryx', principal='oryx/127.15.75.190@KRBTEST.COM'} at 127.0.0.1:50802
I20260809 02:16:23.593870 27892 tablet_replica.cc:330] T fa074991209446d18a9208d8b510fd2f P a1a992abbf44400bbf100e73e082e99f: stopping tablet replica
I20260809 02:16:23.594054 27892 raft_consensus.cc:2229] T fa074991209446d18a9208d8b510fd2f P a1a992abbf44400bbf100e73e082e99f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:16:23.594175 27626 tablet_service.cc:1514] Processing DeleteTablet for tablet fa074991209446d18a9208d8b510fd2f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:16:23 UTC) from {username='oryx', principal='oryx/127.15.75.190@KRBTEST.COM'} at 127.0.0.1:57046
I20260809 02:16:23.594230 27892 raft_consensus.cc:2258] T fa074991209446d18a9208d8b510fd2f P a1a992abbf44400bbf100e73e082e99f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:16:23.595062 27892 ts_tablet_manager.cc:1905] T fa074991209446d18a9208d8b510fd2f P a1a992abbf44400bbf100e73e082e99f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:16:23.598862 27892 ts_tablet_manager.cc:1918] T fa074991209446d18a9208d8b510fd2f P a1a992abbf44400bbf100e73e082e99f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 02:16:23.599022 27892 log.cc:1196] T fa074991209446d18a9208d8b510fd2f P a1a992abbf44400bbf100e73e082e99f: Deleting WAL directory at /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/minicluster-data/ts-0/wal/wals/fa074991209446d18a9208d8b510fd2f
I20260809 02:16:23.599412 27892 ts_tablet_manager.cc:1939] T fa074991209446d18a9208d8b510fd2f P a1a992abbf44400bbf100e73e082e99f: Deleting consensus metadata
I20260809 02:16:23.599646 27895 tablet_replica.cc:330] T fa074991209446d18a9208d8b510fd2f P 621578d59c654ad6b768bf8ff3fd7209: stopping tablet replica
I20260809 02:16:23.600445 26955 catalog_manager.cc:4841] TS a1a992abbf44400bbf100e73e082e99f (127.15.75.129:41883): tablet fa074991209446d18a9208d8b510fd2f (table test-table [id=d8381d37afe441d7a968044f76f6922f]) successfully deleted
I20260809 02:16:23.600764 27894 tablet_replica.cc:330] T fa074991209446d18a9208d8b510fd2f P e33d0f78bb944d67ac12cec28e0bec5b: stopping tablet replica
I20260809 02:16:23.600785 27895 raft_consensus.cc:2229] T fa074991209446d18a9208d8b510fd2f P 621578d59c654ad6b768bf8ff3fd7209 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:16:23.600901 27894 raft_consensus.cc:2229] T fa074991209446d18a9208d8b510fd2f P e33d0f78bb944d67ac12cec28e0bec5b [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:16:23.601052 27895 raft_consensus.cc:2258] T fa074991209446d18a9208d8b510fd2f P 621578d59c654ad6b768bf8ff3fd7209 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:16:23.601207 27894 raft_consensus.cc:2258] T fa074991209446d18a9208d8b510fd2f P e33d0f78bb944d67ac12cec28e0bec5b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:16:23.601816 27895 ts_tablet_manager.cc:1905] T fa074991209446d18a9208d8b510fd2f P 621578d59c654ad6b768bf8ff3fd7209: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:16:23.601817 27894 ts_tablet_manager.cc:1905] T fa074991209446d18a9208d8b510fd2f P e33d0f78bb944d67ac12cec28e0bec5b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:16:23.604669 27894 ts_tablet_manager.cc:1918] T fa074991209446d18a9208d8b510fd2f P e33d0f78bb944d67ac12cec28e0bec5b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 02:16:23.604676 27895 ts_tablet_manager.cc:1918] T fa074991209446d18a9208d8b510fd2f P 621578d59c654ad6b768bf8ff3fd7209: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 02:16:23.604782 27894 log.cc:1196] T fa074991209446d18a9208d8b510fd2f P e33d0f78bb944d67ac12cec28e0bec5b: Deleting WAL directory at /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/minicluster-data/ts-1/wal/wals/fa074991209446d18a9208d8b510fd2f
I20260809 02:16:23.604884 27895 log.cc:1196] T fa074991209446d18a9208d8b510fd2f P 621578d59c654ad6b768bf8ff3fd7209: Deleting WAL directory at /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/minicluster-data/ts-2/wal/wals/fa074991209446d18a9208d8b510fd2f
I20260809 02:16:23.605126 27894 ts_tablet_manager.cc:1939] T fa074991209446d18a9208d8b510fd2f P e33d0f78bb944d67ac12cec28e0bec5b: Deleting consensus metadata
I20260809 02:16:23.605243 27895 ts_tablet_manager.cc:1939] T fa074991209446d18a9208d8b510fd2f P 621578d59c654ad6b768bf8ff3fd7209: Deleting consensus metadata
I20260809 02:16:23.605830 26958 catalog_manager.cc:4841] TS e33d0f78bb944d67ac12cec28e0bec5b (127.15.75.130:44881): tablet fa074991209446d18a9208d8b510fd2f (table test-table [id=d8381d37afe441d7a968044f76f6922f]) successfully deleted
I20260809 02:16:23.605885 26959 catalog_manager.cc:4841] TS 621578d59c654ad6b768bf8ff3fd7209 (127.15.75.131:38591): tablet fa074991209446d18a9208d8b510fd2f (table test-table [id=d8381d37afe441d7a968044f76f6922f]) successfully deleted
Aug 09 02:16:23 dist-test-slave-s11t krb5kdc[26909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241783, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 02:16:23 dist-test-slave-s11t krb5kdc[26909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:16:23 dist-test-slave-s11t krb5kdc[26909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
W20260809 02:16:23.631122 27837 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:16:23.624219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48568 (local address 127.15.75.190:38449)
0809 02:16:23.624602 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:23.624607 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:23.624623 (+ 16us) server_negotiation.cc:407] Connection header received
0809 02:16:23.624862 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:23.624865 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:23.624912 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:23.625001 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:23.625766 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:23.626434 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:23.627391 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:23.627655 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:23.627764 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:23.630988 (+ 3224us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48568: BlockingRecv error: recv got EOF from 127.0.0.1:48568 (error 108)
Metrics: {"server-negotiator.queue_time_us":280}
Aug 09 02:16:23 dist-test-slave-s11t krb5kdc[26909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241783, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM
I20260809 02:16:23.648804 27837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:23.633375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48570 (local address 127.15.75.190:38449)
0809 02:16:23.633516 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:23.633522 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:23.640234 (+ 6712us) server_negotiation.cc:407] Connection header received
0809 02:16:23.640427 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:23.640430 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:23.640468 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:23.640557 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:23.641251 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:23.641765 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:23.642539 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:23.642767 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:23.642874 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:23.645261 (+ 2387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:23.645271 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:23.645275 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:23.645298 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:23.646800 (+ 1502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:23.647309 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:23.647313 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:23.647317 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:23.647398 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:23.647757 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:23.647765 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:23.647767 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:23.647940 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:23.648085 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:23.648387 (+ 302us) server_negotiation.cc:299] Negotiation successful
0809 02:16:23.648577 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 02:16:23.651412 26977 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48570:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:16:23.651885 26977 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:16:23.657080 27626 tablet_service.cc:1467] Processing CreateTablet for tablet 58f0d99626cc4e7b9bb98531a890f9d3 (DEFAULT_TABLE table=test-table [id=45a2ef12fee24faa85a58c6a098dd185]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:16:23.657080 27400 tablet_service.cc:1467] Processing CreateTablet for tablet 58f0d99626cc4e7b9bb98531a890f9d3 (DEFAULT_TABLE table=test-table [id=45a2ef12fee24faa85a58c6a098dd185]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:16:23.657406 27400 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58f0d99626cc4e7b9bb98531a890f9d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:16:23.657414 27626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58f0d99626cc4e7b9bb98531a890f9d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:16:23.659837 27775 tablet_bootstrap.cc:492] T 58f0d99626cc4e7b9bb98531a890f9d3 P e33d0f78bb944d67ac12cec28e0bec5b: Bootstrap starting.
I20260809 02:16:23.660840 27775 tablet_bootstrap.cc:654] T 58f0d99626cc4e7b9bb98531a890f9d3 P e33d0f78bb944d67ac12cec28e0bec5b: Neither blocks nor log segments found. Creating new log.
I20260809 02:16:23.661479 27780 tablet_bootstrap.cc:492] T 58f0d99626cc4e7b9bb98531a890f9d3 P 621578d59c654ad6b768bf8ff3fd7209: Bootstrap starting.
I20260809 02:16:23.662310 27780 tablet_bootstrap.cc:654] T 58f0d99626cc4e7b9bb98531a890f9d3 P 621578d59c654ad6b768bf8ff3fd7209: Neither blocks nor log segments found. Creating new log.
I20260809 02:16:23.663076 27191 tablet_service.cc:1467] Processing CreateTablet for tablet 58f0d99626cc4e7b9bb98531a890f9d3 (DEFAULT_TABLE table=test-table [id=45a2ef12fee24faa85a58c6a098dd185]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:16:23.663383 27191 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58f0d99626cc4e7b9bb98531a890f9d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:16:23.665336 27777 tablet_bootstrap.cc:492] T 58f0d99626cc4e7b9bb98531a890f9d3 P a1a992abbf44400bbf100e73e082e99f: Bootstrap starting.
I20260809 02:16:23.666352 27777 tablet_bootstrap.cc:654] T 58f0d99626cc4e7b9bb98531a890f9d3 P a1a992abbf44400bbf100e73e082e99f: Neither blocks nor log segments found. Creating new log.
I20260809 02:16:23.666504 27780 tablet_bootstrap.cc:492] T 58f0d99626cc4e7b9bb98531a890f9d3 P 621578d59c654ad6b768bf8ff3fd7209: No bootstrap required, opened a new log
I20260809 02:16:23.666584 27780 ts_tablet_manager.cc:1397] T 58f0d99626cc4e7b9bb98531a890f9d3 P 621578d59c654ad6b768bf8ff3fd7209: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.001s
I20260809 02:16:23.667013 27780 raft_consensus.cc:348] T 58f0d99626cc4e7b9bb98531a890f9d3 P 621578d59c654ad6b768bf8ff3fd7209 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a992abbf44400bbf100e73e082e99f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 41883 } } peers { permanent_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 44881 } } peers { permanent_uuid: "621578d59c654ad6b768bf8ff3fd7209" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 38591 } }
I20260809 02:16:23.667136 27780 raft_consensus.cc:374] T 58f0d99626cc4e7b9bb98531a890f9d3 P 621578d59c654ad6b768bf8ff3fd7209 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:16:23.667176 27780 raft_consensus.cc:729] T 58f0d99626cc4e7b9bb98531a890f9d3 P 621578d59c654ad6b768bf8ff3fd7209 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 621578d59c654ad6b768bf8ff3fd7209, State: Initialized, Role: FOLLOWER
I20260809 02:16:23.667300 27780 consensus_queue.cc:260] T 58f0d99626cc4e7b9bb98531a890f9d3 P 621578d59c654ad6b768bf8ff3fd7209 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a992abbf44400bbf100e73e082e99f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 41883 } } peers { permanent_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 44881 } } peers { permanent_uuid: "621578d59c654ad6b768bf8ff3fd7209" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 38591 } }
I20260809 02:16:23.667582 27780 ts_tablet_manager.cc:1428] T 58f0d99626cc4e7b9bb98531a890f9d3 P 621578d59c654ad6b768bf8ff3fd7209: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 02:16:23.668155 27775 tablet_bootstrap.cc:492] T 58f0d99626cc4e7b9bb98531a890f9d3 P e33d0f78bb944d67ac12cec28e0bec5b: No bootstrap required, opened a new log
I20260809 02:16:23.668247 27775 ts_tablet_manager.cc:1397] T 58f0d99626cc4e7b9bb98531a890f9d3 P e33d0f78bb944d67ac12cec28e0bec5b: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.001s
I20260809 02:16:23.668730 27775 raft_consensus.cc:348] T 58f0d99626cc4e7b9bb98531a890f9d3 P e33d0f78bb944d67ac12cec28e0bec5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a992abbf44400bbf100e73e082e99f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 41883 } } peers { permanent_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 44881 } } peers { permanent_uuid: "621578d59c654ad6b768bf8ff3fd7209" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 38591 } }
I20260809 02:16:23.668851 27775 raft_consensus.cc:374] T 58f0d99626cc4e7b9bb98531a890f9d3 P e33d0f78bb944d67ac12cec28e0bec5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:16:23.668886 27775 raft_consensus.cc:729] T 58f0d99626cc4e7b9bb98531a890f9d3 P e33d0f78bb944d67ac12cec28e0bec5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e33d0f78bb944d67ac12cec28e0bec5b, State: Initialized, Role: FOLLOWER
I20260809 02:16:23.669019 27775 consensus_queue.cc:260] T 58f0d99626cc4e7b9bb98531a890f9d3 P e33d0f78bb944d67ac12cec28e0bec5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a992abbf44400bbf100e73e082e99f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 41883 } } peers { permanent_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 44881 } } peers { permanent_uuid: "621578d59c654ad6b768bf8ff3fd7209" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 38591 } }
I20260809 02:16:23.671171 27775 ts_tablet_manager.cc:1428] T 58f0d99626cc4e7b9bb98531a890f9d3 P e33d0f78bb944d67ac12cec28e0bec5b: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20260809 02:16:23.672273 27777 tablet_bootstrap.cc:492] T 58f0d99626cc4e7b9bb98531a890f9d3 P a1a992abbf44400bbf100e73e082e99f: No bootstrap required, opened a new log
I20260809 02:16:23.672364 27777 ts_tablet_manager.cc:1397] T 58f0d99626cc4e7b9bb98531a890f9d3 P a1a992abbf44400bbf100e73e082e99f: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.001s
I20260809 02:16:23.672787 27777 raft_consensus.cc:348] T 58f0d99626cc4e7b9bb98531a890f9d3 P a1a992abbf44400bbf100e73e082e99f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a992abbf44400bbf100e73e082e99f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 41883 } } peers { permanent_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 44881 } } peers { permanent_uuid: "621578d59c654ad6b768bf8ff3fd7209" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 38591 } }
I20260809 02:16:23.672917 27777 raft_consensus.cc:374] T 58f0d99626cc4e7b9bb98531a890f9d3 P a1a992abbf44400bbf100e73e082e99f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:16:23.672955 27777 raft_consensus.cc:729] T 58f0d99626cc4e7b9bb98531a890f9d3 P a1a992abbf44400bbf100e73e082e99f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1a992abbf44400bbf100e73e082e99f, State: Initialized, Role: FOLLOWER
I20260809 02:16:23.673071 27777 consensus_queue.cc:260] T 58f0d99626cc4e7b9bb98531a890f9d3 P a1a992abbf44400bbf100e73e082e99f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a992abbf44400bbf100e73e082e99f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 41883 } } peers { permanent_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 44881 } } peers { permanent_uuid: "621578d59c654ad6b768bf8ff3fd7209" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 38591 } }
I20260809 02:16:23.673365 27777 ts_tablet_manager.cc:1428] T 58f0d99626cc4e7b9bb98531a890f9d3 P a1a992abbf44400bbf100e73e082e99f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 02:16:23.781774 27825 raft_consensus.cc:482] T 58f0d99626cc4e7b9bb98531a890f9d3 P e33d0f78bb944d67ac12cec28e0bec5b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:16:23.781929 27825 raft_consensus.cc:504] T 58f0d99626cc4e7b9bb98531a890f9d3 P e33d0f78bb944d67ac12cec28e0bec5b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a992abbf44400bbf100e73e082e99f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 41883 } } peers { permanent_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 44881 } } peers { permanent_uuid: "621578d59c654ad6b768bf8ff3fd7209" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 38591 } }
I20260809 02:16:23.782377 27825 leader_election.cc:288] T 58f0d99626cc4e7b9bb98531a890f9d3 P e33d0f78bb944d67ac12cec28e0bec5b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a1a992abbf44400bbf100e73e082e99f (127.15.75.129:41883), 621578d59c654ad6b768bf8ff3fd7209 (127.15.75.131:38591)
I20260809 02:16:23.782833 27656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58f0d99626cc4e7b9bb98531a890f9d3" candidate_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "621578d59c654ad6b768bf8ff3fd7209" is_pre_election: true
I20260809 02:16:23.782852 27213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58f0d99626cc4e7b9bb98531a890f9d3" candidate_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1a992abbf44400bbf100e73e082e99f" is_pre_election: true
I20260809 02:16:23.782979 27656 raft_consensus.cc:2454] T 58f0d99626cc4e7b9bb98531a890f9d3 P 621578d59c654ad6b768bf8ff3fd7209 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e33d0f78bb944d67ac12cec28e0bec5b in term 0.
I20260809 02:16:23.783027 27213 raft_consensus.cc:2454] T 58f0d99626cc4e7b9bb98531a890f9d3 P a1a992abbf44400bbf100e73e082e99f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e33d0f78bb944d67ac12cec28e0bec5b in term 0.
I20260809 02:16:23.783310 27335 leader_election.cc:302] T 58f0d99626cc4e7b9bb98531a890f9d3 P e33d0f78bb944d67ac12cec28e0bec5b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 621578d59c654ad6b768bf8ff3fd7209, e33d0f78bb944d67ac12cec28e0bec5b; no voters:
I20260809 02:16:23.783501 27825 raft_consensus.cc:2781] T 58f0d99626cc4e7b9bb98531a890f9d3 P e33d0f78bb944d67ac12cec28e0bec5b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:16:23.783560 27825 raft_consensus.cc:482] T 58f0d99626cc4e7b9bb98531a890f9d3 P e33d0f78bb944d67ac12cec28e0bec5b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:16:23.783587 27825 raft_consensus.cc:3037] T 58f0d99626cc4e7b9bb98531a890f9d3 P e33d0f78bb944d67ac12cec28e0bec5b [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:16:23.784530 27825 raft_consensus.cc:504] T 58f0d99626cc4e7b9bb98531a890f9d3 P e33d0f78bb944d67ac12cec28e0bec5b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a992abbf44400bbf100e73e082e99f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 41883 } } peers { permanent_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 44881 } } peers { permanent_uuid: "621578d59c654ad6b768bf8ff3fd7209" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 38591 } }
I20260809 02:16:23.784937 27825 leader_election.cc:288] T 58f0d99626cc4e7b9bb98531a890f9d3 P e33d0f78bb944d67ac12cec28e0bec5b [CANDIDATE]: Term 1 election: Requested vote from peers a1a992abbf44400bbf100e73e082e99f (127.15.75.129:41883), 621578d59c654ad6b768bf8ff3fd7209 (127.15.75.131:38591)
I20260809 02:16:23.785269 27213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58f0d99626cc4e7b9bb98531a890f9d3" candidate_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1a992abbf44400bbf100e73e082e99f"
I20260809 02:16:23.785355 27656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58f0d99626cc4e7b9bb98531a890f9d3" candidate_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "621578d59c654ad6b768bf8ff3fd7209"
I20260809 02:16:23.785420 27213 raft_consensus.cc:3037] T 58f0d99626cc4e7b9bb98531a890f9d3 P a1a992abbf44400bbf100e73e082e99f [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:16:23.785501 27656 raft_consensus.cc:3037] T 58f0d99626cc4e7b9bb98531a890f9d3 P 621578d59c654ad6b768bf8ff3fd7209 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:16:23.786367 27213 raft_consensus.cc:2454] T 58f0d99626cc4e7b9bb98531a890f9d3 P a1a992abbf44400bbf100e73e082e99f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e33d0f78bb944d67ac12cec28e0bec5b in term 1.
I20260809 02:16:23.786466 27656 raft_consensus.cc:2454] T 58f0d99626cc4e7b9bb98531a890f9d3 P 621578d59c654ad6b768bf8ff3fd7209 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e33d0f78bb944d67ac12cec28e0bec5b in term 1.
I20260809 02:16:23.786655 27328 leader_election.cc:302] T 58f0d99626cc4e7b9bb98531a890f9d3 P e33d0f78bb944d67ac12cec28e0bec5b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a1a992abbf44400bbf100e73e082e99f, e33d0f78bb944d67ac12cec28e0bec5b; no voters:
I20260809 02:16:23.786860 27825 raft_consensus.cc:2781] T 58f0d99626cc4e7b9bb98531a890f9d3 P e33d0f78bb944d67ac12cec28e0bec5b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:16:23.786966 27825 raft_consensus.cc:686] T 58f0d99626cc4e7b9bb98531a890f9d3 P e33d0f78bb944d67ac12cec28e0bec5b [term 1 LEADER]: Becoming Leader. State: Replica: e33d0f78bb944d67ac12cec28e0bec5b, State: Running, Role: LEADER
I20260809 02:16:23.787113 27825 consensus_queue.cc:237] T 58f0d99626cc4e7b9bb98531a890f9d3 P e33d0f78bb944d67ac12cec28e0bec5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a992abbf44400bbf100e73e082e99f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 41883 } } peers { permanent_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 44881 } } peers { permanent_uuid: "621578d59c654ad6b768bf8ff3fd7209" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 38591 } }
I20260809 02:16:23.788970 26974 catalog_manager.cc:5495] T 58f0d99626cc4e7b9bb98531a890f9d3 P e33d0f78bb944d67ac12cec28e0bec5b reported cstate change: term changed from 0 to 1, leader changed from <none> to e33d0f78bb944d67ac12cec28e0bec5b (127.15.75.130). New cstate: current_term: 1 leader_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a992abbf44400bbf100e73e082e99f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 41883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 44881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "621578d59c654ad6b768bf8ff3fd7209" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 38591 } health_report { overall_health: UNKNOWN } } }
I20260809 02:16:23.826471 27870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:23.823041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50820 (local address 127.15.75.130:44881)
0809 02:16:23.823194 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:23.823200 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:23.823214 (+ 14us) server_negotiation.cc:407] Connection header received
0809 02:16:23.823306 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:23.823309 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:23.823361 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:23.823447 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:16:23.823700 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:23.824238 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:23.825264 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:23.825497 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:23.825601 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:23.825937 (+ 336us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:16:23.826024 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:16:23.826097 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:23.826207 (+ 110us) server_negotiation.cc:299] Negotiation successful
0809 02:16:23.826299 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 02:16:23.828670 27213 raft_consensus.cc:1261] T 58f0d99626cc4e7b9bb98531a890f9d3 P a1a992abbf44400bbf100e73e082e99f [term 1 FOLLOWER]: Refusing update from remote peer e33d0f78bb944d67ac12cec28e0bec5b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:16:23.828716 27656 raft_consensus.cc:1261] T 58f0d99626cc4e7b9bb98531a890f9d3 P 621578d59c654ad6b768bf8ff3fd7209 [term 1 FOLLOWER]: Refusing update from remote peer e33d0f78bb944d67ac12cec28e0bec5b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:16:23.829186 27825 consensus_queue.cc:1035] T 58f0d99626cc4e7b9bb98531a890f9d3 P e33d0f78bb944d67ac12cec28e0bec5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1a992abbf44400bbf100e73e082e99f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 41883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:16:23.829344 27888 consensus_queue.cc:1035] T 58f0d99626cc4e7b9bb98531a890f9d3 P e33d0f78bb944d67ac12cec28e0bec5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "621578d59c654ad6b768bf8ff3fd7209" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 38591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:16:23.836398 26974 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48570:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:16:23.836555 26974 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48570:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:16:23.838507 26974 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 34bc02e2aabc462baa95d3fc45949cba: Sending DeleteTablet for 3 replicas of tablet 58f0d99626cc4e7b9bb98531a890f9d3
I20260809 02:16:23.839077 27400 tablet_service.cc:1514] Processing DeleteTablet for tablet 58f0d99626cc4e7b9bb98531a890f9d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:16:23 UTC) from {username='oryx', principal='oryx/127.15.75.190@KRBTEST.COM'} at 127.0.0.1:50802
I20260809 02:16:23.839102 27191 tablet_service.cc:1514] Processing DeleteTablet for tablet 58f0d99626cc4e7b9bb98531a890f9d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:16:23 UTC) from {username='oryx', principal='oryx/127.15.75.190@KRBTEST.COM'} at 127.0.0.1:40740
I20260809 02:16:23.839242 27894 tablet_replica.cc:330] T 58f0d99626cc4e7b9bb98531a890f9d3 P e33d0f78bb944d67ac12cec28e0bec5b: stopping tablet replica
I20260809 02:16:23.839351 27894 raft_consensus.cc:2229] T 58f0d99626cc4e7b9bb98531a890f9d3 P e33d0f78bb944d67ac12cec28e0bec5b [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:16:23.839461 27892 tablet_replica.cc:330] T 58f0d99626cc4e7b9bb98531a890f9d3 P a1a992abbf44400bbf100e73e082e99f: stopping tablet replica
I20260809 02:16:23.839540 27894 raft_consensus.cc:2258] T 58f0d99626cc4e7b9bb98531a890f9d3 P e33d0f78bb944d67ac12cec28e0bec5b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:16:23.839571 27892 raft_consensus.cc:2229] T 58f0d99626cc4e7b9bb98531a890f9d3 P a1a992abbf44400bbf100e73e082e99f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:16:23.839679 27892 raft_consensus.cc:2258] T 58f0d99626cc4e7b9bb98531a890f9d3 P a1a992abbf44400bbf100e73e082e99f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:16:23.839716 27626 tablet_service.cc:1514] Processing DeleteTablet for tablet 58f0d99626cc4e7b9bb98531a890f9d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:16:23 UTC) from {username='oryx', principal='oryx/127.15.75.190@KRBTEST.COM'} at 127.0.0.1:57046
I20260809 02:16:23.839874 27895 tablet_replica.cc:330] T 58f0d99626cc4e7b9bb98531a890f9d3 P 621578d59c654ad6b768bf8ff3fd7209: stopping tablet replica
I20260809 02:16:23.839986 27895 raft_consensus.cc:2229] T 58f0d99626cc4e7b9bb98531a890f9d3 P 621578d59c654ad6b768bf8ff3fd7209 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:16:23.840106 27895 raft_consensus.cc:2258] T 58f0d99626cc4e7b9bb98531a890f9d3 P 621578d59c654ad6b768bf8ff3fd7209 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:16:23.840209 26974 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48570:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 02:16:23.840348 27892 ts_tablet_manager.cc:1905] T 58f0d99626cc4e7b9bb98531a890f9d3 P a1a992abbf44400bbf100e73e082e99f: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 02:16:23.840785 26974 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:16:23.840934 27894 ts_tablet_manager.cc:1905] T 58f0d99626cc4e7b9bb98531a890f9d3 P e33d0f78bb944d67ac12cec28e0bec5b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:16:23.841897 27895 ts_tablet_manager.cc:1905] T 58f0d99626cc4e7b9bb98531a890f9d3 P 621578d59c654ad6b768bf8ff3fd7209: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:16:23.842980 27892 ts_tablet_manager.cc:1918] T 58f0d99626cc4e7b9bb98531a890f9d3 P a1a992abbf44400bbf100e73e082e99f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:16:23.843066 27892 log.cc:1196] T 58f0d99626cc4e7b9bb98531a890f9d3 P a1a992abbf44400bbf100e73e082e99f: Deleting WAL directory at /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/minicluster-data/ts-0/wal/wals/58f0d99626cc4e7b9bb98531a890f9d3
I20260809 02:16:23.843134 27894 ts_tablet_manager.cc:1918] T 58f0d99626cc4e7b9bb98531a890f9d3 P e33d0f78bb944d67ac12cec28e0bec5b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:16:23.843214 27894 log.cc:1196] T 58f0d99626cc4e7b9bb98531a890f9d3 P e33d0f78bb944d67ac12cec28e0bec5b: Deleting WAL directory at /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/minicluster-data/ts-1/wal/wals/58f0d99626cc4e7b9bb98531a890f9d3
I20260809 02:16:23.843340 27892 ts_tablet_manager.cc:1939] T 58f0d99626cc4e7b9bb98531a890f9d3 P a1a992abbf44400bbf100e73e082e99f: Deleting consensus metadata
I20260809 02:16:23.843546 27894 ts_tablet_manager.cc:1939] T 58f0d99626cc4e7b9bb98531a890f9d3 P e33d0f78bb944d67ac12cec28e0bec5b: Deleting consensus metadata
I20260809 02:16:23.844053 26955 catalog_manager.cc:4841] TS a1a992abbf44400bbf100e73e082e99f (127.15.75.129:41883): tablet 58f0d99626cc4e7b9bb98531a890f9d3 (table test-table [id=45a2ef12fee24faa85a58c6a098dd185]) successfully deleted
I20260809 02:16:23.844201 26958 catalog_manager.cc:4841] TS e33d0f78bb944d67ac12cec28e0bec5b (127.15.75.130:44881): tablet 58f0d99626cc4e7b9bb98531a890f9d3 (table test-table [id=45a2ef12fee24faa85a58c6a098dd185]) successfully deleted
I20260809 02:16:23.844419 27895 ts_tablet_manager.cc:1918] T 58f0d99626cc4e7b9bb98531a890f9d3 P 621578d59c654ad6b768bf8ff3fd7209: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:16:23.844517 27895 log.cc:1196] T 58f0d99626cc4e7b9bb98531a890f9d3 P 621578d59c654ad6b768bf8ff3fd7209: Deleting WAL directory at /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/minicluster-data/ts-2/wal/wals/58f0d99626cc4e7b9bb98531a890f9d3
I20260809 02:16:23.844841 27895 ts_tablet_manager.cc:1939] T 58f0d99626cc4e7b9bb98531a890f9d3 P 621578d59c654ad6b768bf8ff3fd7209: Deleting consensus metadata
I20260809 02:16:23.845429 26959 catalog_manager.cc:4841] TS 621578d59c654ad6b768bf8ff3fd7209 (127.15.75.131:38591): tablet 58f0d99626cc4e7b9bb98531a890f9d3 (table test-table [id=45a2ef12fee24faa85a58c6a098dd185]) successfully deleted
I20260809 02:16:23.847087 27191 tablet_service.cc:1467] Processing CreateTablet for tablet e09e1ef767fd4b57b3a9a464e9029610 (DEFAULT_TABLE table=test-table [id=f0407d6e6cff4ad1b2e2a887b163317f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:16:23.847158 27400 tablet_service.cc:1467] Processing CreateTablet for tablet e09e1ef767fd4b57b3a9a464e9029610 (DEFAULT_TABLE table=test-table [id=f0407d6e6cff4ad1b2e2a887b163317f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:16:23.847350 27191 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e09e1ef767fd4b57b3a9a464e9029610. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:16:23.847307 27626 tablet_service.cc:1467] Processing CreateTablet for tablet e09e1ef767fd4b57b3a9a464e9029610 (DEFAULT_TABLE table=test-table [id=f0407d6e6cff4ad1b2e2a887b163317f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:16:23.847393 27400 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e09e1ef767fd4b57b3a9a464e9029610. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:16:23.847563 27626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e09e1ef767fd4b57b3a9a464e9029610. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:16:23.849529 27775 tablet_bootstrap.cc:492] T e09e1ef767fd4b57b3a9a464e9029610 P e33d0f78bb944d67ac12cec28e0bec5b: Bootstrap starting.
I20260809 02:16:23.849529 27777 tablet_bootstrap.cc:492] T e09e1ef767fd4b57b3a9a464e9029610 P a1a992abbf44400bbf100e73e082e99f: Bootstrap starting.
I20260809 02:16:23.849694 27780 tablet_bootstrap.cc:492] T e09e1ef767fd4b57b3a9a464e9029610 P 621578d59c654ad6b768bf8ff3fd7209: Bootstrap starting.
I20260809 02:16:23.850313 27777 tablet_bootstrap.cc:654] T e09e1ef767fd4b57b3a9a464e9029610 P a1a992abbf44400bbf100e73e082e99f: Neither blocks nor log segments found. Creating new log.
I20260809 02:16:23.850313 27775 tablet_bootstrap.cc:654] T e09e1ef767fd4b57b3a9a464e9029610 P e33d0f78bb944d67ac12cec28e0bec5b: Neither blocks nor log segments found. Creating new log.
I20260809 02:16:23.850342 27780 tablet_bootstrap.cc:654] T e09e1ef767fd4b57b3a9a464e9029610 P 621578d59c654ad6b768bf8ff3fd7209: Neither blocks nor log segments found. Creating new log.
I20260809 02:16:23.851315 27780 tablet_bootstrap.cc:492] T e09e1ef767fd4b57b3a9a464e9029610 P 621578d59c654ad6b768bf8ff3fd7209: No bootstrap required, opened a new log
I20260809 02:16:23.851423 27780 ts_tablet_manager.cc:1397] T e09e1ef767fd4b57b3a9a464e9029610 P 621578d59c654ad6b768bf8ff3fd7209: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260809 02:16:23.851492 27775 tablet_bootstrap.cc:492] T e09e1ef767fd4b57b3a9a464e9029610 P e33d0f78bb944d67ac12cec28e0bec5b: No bootstrap required, opened a new log
I20260809 02:16:23.851559 27775 ts_tablet_manager.cc:1397] T e09e1ef767fd4b57b3a9a464e9029610 P e33d0f78bb944d67ac12cec28e0bec5b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 02:16:23.851735 27777 tablet_bootstrap.cc:492] T e09e1ef767fd4b57b3a9a464e9029610 P a1a992abbf44400bbf100e73e082e99f: No bootstrap required, opened a new log
I20260809 02:16:23.851810 27777 ts_tablet_manager.cc:1397] T e09e1ef767fd4b57b3a9a464e9029610 P a1a992abbf44400bbf100e73e082e99f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 02:16:23.851897 27775 raft_consensus.cc:348] T e09e1ef767fd4b57b3a9a464e9029610 P e33d0f78bb944d67ac12cec28e0bec5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a992abbf44400bbf100e73e082e99f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 41883 } } peers { permanent_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 44881 } } peers { permanent_uuid: "621578d59c654ad6b768bf8ff3fd7209" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 38591 } }
I20260809 02:16:23.851976 27775 raft_consensus.cc:374] T e09e1ef767fd4b57b3a9a464e9029610 P e33d0f78bb944d67ac12cec28e0bec5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:16:23.851997 27775 raft_consensus.cc:729] T e09e1ef767fd4b57b3a9a464e9029610 P e33d0f78bb944d67ac12cec28e0bec5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e33d0f78bb944d67ac12cec28e0bec5b, State: Initialized, Role: FOLLOWER
I20260809 02:16:23.851821 27780 raft_consensus.cc:348] T e09e1ef767fd4b57b3a9a464e9029610 P 621578d59c654ad6b768bf8ff3fd7209 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a992abbf44400bbf100e73e082e99f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 41883 } } peers { permanent_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 44881 } } peers { permanent_uuid: "621578d59c654ad6b768bf8ff3fd7209" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 38591 } }
I20260809 02:16:23.852041 27780 raft_consensus.cc:374] T e09e1ef767fd4b57b3a9a464e9029610 P 621578d59c654ad6b768bf8ff3fd7209 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:16:23.852082 27775 consensus_queue.cc:260] T e09e1ef767fd4b57b3a9a464e9029610 P e33d0f78bb944d67ac12cec28e0bec5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a992abbf44400bbf100e73e082e99f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 41883 } } peers { permanent_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 44881 } } peers { permanent_uuid: "621578d59c654ad6b768bf8ff3fd7209" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 38591 } }
I20260809 02:16:23.852098 27780 raft_consensus.cc:729] T e09e1ef767fd4b57b3a9a464e9029610 P 621578d59c654ad6b768bf8ff3fd7209 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 621578d59c654ad6b768bf8ff3fd7209, State: Initialized, Role: FOLLOWER
I20260809 02:16:23.852258 27775 ts_tablet_manager.cc:1428] T e09e1ef767fd4b57b3a9a464e9029610 P e33d0f78bb944d67ac12cec28e0bec5b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 02:16:23.852222 27780 consensus_queue.cc:260] T e09e1ef767fd4b57b3a9a464e9029610 P 621578d59c654ad6b768bf8ff3fd7209 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a992abbf44400bbf100e73e082e99f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 41883 } } peers { permanent_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 44881 } } peers { permanent_uuid: "621578d59c654ad6b768bf8ff3fd7209" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 38591 } }
I20260809 02:16:23.852311 27777 raft_consensus.cc:348] T e09e1ef767fd4b57b3a9a464e9029610 P a1a992abbf44400bbf100e73e082e99f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a992abbf44400bbf100e73e082e99f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 41883 } } peers { permanent_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 44881 } } peers { permanent_uuid: "621578d59c654ad6b768bf8ff3fd7209" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 38591 } }
I20260809 02:16:23.852411 27777 raft_consensus.cc:374] T e09e1ef767fd4b57b3a9a464e9029610 P a1a992abbf44400bbf100e73e082e99f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:16:23.852442 27777 raft_consensus.cc:729] T e09e1ef767fd4b57b3a9a464e9029610 P a1a992abbf44400bbf100e73e082e99f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1a992abbf44400bbf100e73e082e99f, State: Initialized, Role: FOLLOWER
I20260809 02:16:23.852514 27780 ts_tablet_manager.cc:1428] T e09e1ef767fd4b57b3a9a464e9029610 P 621578d59c654ad6b768bf8ff3fd7209: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 02:16:23.852561 27777 consensus_queue.cc:260] T e09e1ef767fd4b57b3a9a464e9029610 P a1a992abbf44400bbf100e73e082e99f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a992abbf44400bbf100e73e082e99f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 41883 } } peers { permanent_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 44881 } } peers { permanent_uuid: "621578d59c654ad6b768bf8ff3fd7209" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 38591 } }
I20260809 02:16:23.852856 27777 ts_tablet_manager.cc:1428] T e09e1ef767fd4b57b3a9a464e9029610 P a1a992abbf44400bbf100e73e082e99f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 02:16:24.106598 27783 raft_consensus.cc:482] T e09e1ef767fd4b57b3a9a464e9029610 P e33d0f78bb944d67ac12cec28e0bec5b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:16:24.106734 27783 raft_consensus.cc:504] T e09e1ef767fd4b57b3a9a464e9029610 P e33d0f78bb944d67ac12cec28e0bec5b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a992abbf44400bbf100e73e082e99f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 41883 } } peers { permanent_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 44881 } } peers { permanent_uuid: "621578d59c654ad6b768bf8ff3fd7209" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 38591 } }
I20260809 02:16:24.107211 27783 leader_election.cc:288] T e09e1ef767fd4b57b3a9a464e9029610 P e33d0f78bb944d67ac12cec28e0bec5b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a1a992abbf44400bbf100e73e082e99f (127.15.75.129:41883), 621578d59c654ad6b768bf8ff3fd7209 (127.15.75.131:38591)
I20260809 02:16:24.107667 27213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e09e1ef767fd4b57b3a9a464e9029610" candidate_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1a992abbf44400bbf100e73e082e99f" is_pre_election: true
I20260809 02:16:24.107667 27656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e09e1ef767fd4b57b3a9a464e9029610" candidate_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "621578d59c654ad6b768bf8ff3fd7209" is_pre_election: true
I20260809 02:16:24.107833 27213 raft_consensus.cc:2454] T e09e1ef767fd4b57b3a9a464e9029610 P a1a992abbf44400bbf100e73e082e99f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e33d0f78bb944d67ac12cec28e0bec5b in term 0.
I20260809 02:16:24.107844 27656 raft_consensus.cc:2454] T e09e1ef767fd4b57b3a9a464e9029610 P 621578d59c654ad6b768bf8ff3fd7209 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e33d0f78bb944d67ac12cec28e0bec5b in term 0.
I20260809 02:16:24.108242 27328 leader_election.cc:302] T e09e1ef767fd4b57b3a9a464e9029610 P e33d0f78bb944d67ac12cec28e0bec5b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a1a992abbf44400bbf100e73e082e99f, e33d0f78bb944d67ac12cec28e0bec5b; no voters:
I20260809 02:16:24.108451 27783 raft_consensus.cc:2781] T e09e1ef767fd4b57b3a9a464e9029610 P e33d0f78bb944d67ac12cec28e0bec5b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:16:24.108541 27783 raft_consensus.cc:482] T e09e1ef767fd4b57b3a9a464e9029610 P e33d0f78bb944d67ac12cec28e0bec5b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:16:24.108596 27783 raft_consensus.cc:3037] T e09e1ef767fd4b57b3a9a464e9029610 P e33d0f78bb944d67ac12cec28e0bec5b [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:16:24.109642 27783 raft_consensus.cc:504] T e09e1ef767fd4b57b3a9a464e9029610 P e33d0f78bb944d67ac12cec28e0bec5b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a992abbf44400bbf100e73e082e99f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 41883 } } peers { permanent_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 44881 } } peers { permanent_uuid: "621578d59c654ad6b768bf8ff3fd7209" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 38591 } }
I20260809 02:16:24.109999 27783 leader_election.cc:288] T e09e1ef767fd4b57b3a9a464e9029610 P e33d0f78bb944d67ac12cec28e0bec5b [CANDIDATE]: Term 1 election: Requested vote from peers a1a992abbf44400bbf100e73e082e99f (127.15.75.129:41883), 621578d59c654ad6b768bf8ff3fd7209 (127.15.75.131:38591)
I20260809 02:16:24.110375 27656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e09e1ef767fd4b57b3a9a464e9029610" candidate_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "621578d59c654ad6b768bf8ff3fd7209"
I20260809 02:16:24.110424 27213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e09e1ef767fd4b57b3a9a464e9029610" candidate_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1a992abbf44400bbf100e73e082e99f"
I20260809 02:16:24.110487 27656 raft_consensus.cc:3037] T e09e1ef767fd4b57b3a9a464e9029610 P 621578d59c654ad6b768bf8ff3fd7209 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:16:24.110591 27213 raft_consensus.cc:3037] T e09e1ef767fd4b57b3a9a464e9029610 P a1a992abbf44400bbf100e73e082e99f [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:16:24.111487 27656 raft_consensus.cc:2454] T e09e1ef767fd4b57b3a9a464e9029610 P 621578d59c654ad6b768bf8ff3fd7209 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e33d0f78bb944d67ac12cec28e0bec5b in term 1.
I20260809 02:16:24.111500 27213 raft_consensus.cc:2454] T e09e1ef767fd4b57b3a9a464e9029610 P a1a992abbf44400bbf100e73e082e99f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e33d0f78bb944d67ac12cec28e0bec5b in term 1.
I20260809 02:16:24.111802 27328 leader_election.cc:302] T e09e1ef767fd4b57b3a9a464e9029610 P e33d0f78bb944d67ac12cec28e0bec5b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a1a992abbf44400bbf100e73e082e99f, e33d0f78bb944d67ac12cec28e0bec5b; no voters:
I20260809 02:16:24.112039 27783 raft_consensus.cc:2781] T e09e1ef767fd4b57b3a9a464e9029610 P e33d0f78bb944d67ac12cec28e0bec5b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:16:24.112161 27783 raft_consensus.cc:686] T e09e1ef767fd4b57b3a9a464e9029610 P e33d0f78bb944d67ac12cec28e0bec5b [term 1 LEADER]: Becoming Leader. State: Replica: e33d0f78bb944d67ac12cec28e0bec5b, State: Running, Role: LEADER
I20260809 02:16:24.112316 27783 consensus_queue.cc:237] T e09e1ef767fd4b57b3a9a464e9029610 P e33d0f78bb944d67ac12cec28e0bec5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a992abbf44400bbf100e73e082e99f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 41883 } } peers { permanent_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 44881 } } peers { permanent_uuid: "621578d59c654ad6b768bf8ff3fd7209" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 38591 } }
I20260809 02:16:24.114181 26975 catalog_manager.cc:5495] T e09e1ef767fd4b57b3a9a464e9029610 P e33d0f78bb944d67ac12cec28e0bec5b reported cstate change: term changed from 0 to 1, leader changed from <none> to e33d0f78bb944d67ac12cec28e0bec5b (127.15.75.130). New cstate: current_term: 1 leader_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1a992abbf44400bbf100e73e082e99f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 41883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e33d0f78bb944d67ac12cec28e0bec5b" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 44881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "621578d59c654ad6b768bf8ff3fd7209" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 38591 } health_report { overall_health: UNKNOWN } } }
I20260809 02:16:24.153023 27656 raft_consensus.cc:1261] T e09e1ef767fd4b57b3a9a464e9029610 P 621578d59c654ad6b768bf8ff3fd7209 [term 1 FOLLOWER]: Refusing update from remote peer e33d0f78bb944d67ac12cec28e0bec5b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:16:24.153053 27213 raft_consensus.cc:1261] T e09e1ef767fd4b57b3a9a464e9029610 P a1a992abbf44400bbf100e73e082e99f [term 1 FOLLOWER]: Refusing update from remote peer e33d0f78bb944d67ac12cec28e0bec5b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:16:24.153419 27783 consensus_queue.cc:1035] T e09e1ef767fd4b57b3a9a464e9029610 P e33d0f78bb944d67ac12cec28e0bec5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "621578d59c654ad6b768bf8ff3fd7209" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 38591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:16:24.153574 27825 consensus_queue.cc:1035] T e09e1ef767fd4b57b3a9a464e9029610 P e33d0f78bb944d67ac12cec28e0bec5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1a992abbf44400bbf100e73e082e99f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 41883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:16:24.185704 26974 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48570:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:16:24.185874 26974 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48570:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:16:24.187808 26974 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 34bc02e2aabc462baa95d3fc45949cba: Sending DeleteTablet for 3 replicas of tablet e09e1ef767fd4b57b3a9a464e9029610
I20260809 02:16:24.188351 27400 tablet_service.cc:1514] Processing DeleteTablet for tablet e09e1ef767fd4b57b3a9a464e9029610 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:16:24 UTC) from {username='oryx', principal='oryx/127.15.75.190@KRBTEST.COM'} at 127.0.0.1:50802
I20260809 02:16:24.188374 27191 tablet_service.cc:1514] Processing DeleteTablet for tablet e09e1ef767fd4b57b3a9a464e9029610 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:16:24 UTC) from {username='oryx', principal='oryx/127.15.75.190@KRBTEST.COM'} at 127.0.0.1:40740
I20260809 02:16:24.188413 27626 tablet_service.cc:1514] Processing DeleteTablet for tablet e09e1ef767fd4b57b3a9a464e9029610 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:16:24 UTC) from {username='oryx', principal='oryx/127.15.75.190@KRBTEST.COM'} at 127.0.0.1:57046
I20260809 02:16:24.188532 27894 tablet_replica.cc:330] T e09e1ef767fd4b57b3a9a464e9029610 P e33d0f78bb944d67ac12cec28e0bec5b: stopping tablet replica
I20260809 02:16:24.188550 27892 tablet_replica.cc:330] T e09e1ef767fd4b57b3a9a464e9029610 P a1a992abbf44400bbf100e73e082e99f: stopping tablet replica
I20260809 02:16:24.188566 27895 tablet_replica.cc:330] T e09e1ef767fd4b57b3a9a464e9029610 P 621578d59c654ad6b768bf8ff3fd7209: stopping tablet replica
I20260809 02:16:24.188640 27894 raft_consensus.cc:2229] T e09e1ef767fd4b57b3a9a464e9029610 P e33d0f78bb944d67ac12cec28e0bec5b [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:16:24.188658 27892 raft_consensus.cc:2229] T e09e1ef767fd4b57b3a9a464e9029610 P a1a992abbf44400bbf100e73e082e99f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:16:24.188666 27895 raft_consensus.cc:2229] T e09e1ef767fd4b57b3a9a464e9029610 P 621578d59c654ad6b768bf8ff3fd7209 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:16:24.188792 27892 raft_consensus.cc:2258] T e09e1ef767fd4b57b3a9a464e9029610 P a1a992abbf44400bbf100e73e082e99f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:16:24.188832 27895 raft_consensus.cc:2258] T e09e1ef767fd4b57b3a9a464e9029610 P 621578d59c654ad6b768bf8ff3fd7209 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:16:24.188833 27894 raft_consensus.cc:2258] T e09e1ef767fd4b57b3a9a464e9029610 P e33d0f78bb944d67ac12cec28e0bec5b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:16:24.189568 27895 ts_tablet_manager.cc:1905] T e09e1ef767fd4b57b3a9a464e9029610 P 621578d59c654ad6b768bf8ff3fd7209: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:16:24.189654 27892 ts_tablet_manager.cc:1905] T e09e1ef767fd4b57b3a9a464e9029610 P a1a992abbf44400bbf100e73e082e99f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:16:24.189745 27894 ts_tablet_manager.cc:1905] T e09e1ef767fd4b57b3a9a464e9029610 P e33d0f78bb944d67ac12cec28e0bec5b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:16:24.191771 27895 ts_tablet_manager.cc:1918] T e09e1ef767fd4b57b3a9a464e9029610 P 621578d59c654ad6b768bf8ff3fd7209: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 02:16:24.191871 27895 log.cc:1196] T e09e1ef767fd4b57b3a9a464e9029610 P 621578d59c654ad6b768bf8ff3fd7209: Deleting WAL directory at /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/minicluster-data/ts-2/wal/wals/e09e1ef767fd4b57b3a9a464e9029610
I20260809 02:16:24.192088 27892 ts_tablet_manager.cc:1918] T e09e1ef767fd4b57b3a9a464e9029610 P a1a992abbf44400bbf100e73e082e99f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 02:16:24.192088 27894 ts_tablet_manager.cc:1918] T e09e1ef767fd4b57b3a9a464e9029610 P e33d0f78bb944d67ac12cec28e0bec5b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 02:16:24.192171 27892 log.cc:1196] T e09e1ef767fd4b57b3a9a464e9029610 P a1a992abbf44400bbf100e73e082e99f: Deleting WAL directory at /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/minicluster-data/ts-0/wal/wals/e09e1ef767fd4b57b3a9a464e9029610
I20260809 02:16:24.192171 27894 log.cc:1196] T e09e1ef767fd4b57b3a9a464e9029610 P e33d0f78bb944d67ac12cec28e0bec5b: Deleting WAL directory at /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241741788589-15662-0/minicluster-data/ts-1/wal/wals/e09e1ef767fd4b57b3a9a464e9029610
I20260809 02:16:24.192189 27895 ts_tablet_manager.cc:1939] T e09e1ef767fd4b57b3a9a464e9029610 P 621578d59c654ad6b768bf8ff3fd7209: Deleting consensus metadata
I20260809 02:16:24.192445 27894 ts_tablet_manager.cc:1939] T e09e1ef767fd4b57b3a9a464e9029610 P e33d0f78bb944d67ac12cec28e0bec5b: Deleting consensus metadata
I20260809 02:16:24.192445 27892 ts_tablet_manager.cc:1939] T e09e1ef767fd4b57b3a9a464e9029610 P a1a992abbf44400bbf100e73e082e99f: Deleting consensus metadata
I20260809 02:16:24.192862 26959 catalog_manager.cc:4841] TS 621578d59c654ad6b768bf8ff3fd7209 (127.15.75.131:38591): tablet e09e1ef767fd4b57b3a9a464e9029610 (table test-table [id=f0407d6e6cff4ad1b2e2a887b163317f]) successfully deleted
I20260809 02:16:24.193070 26958 catalog_manager.cc:4841] TS e33d0f78bb944d67ac12cec28e0bec5b (127.15.75.130:44881): tablet e09e1ef767fd4b57b3a9a464e9029610 (table test-table [id=f0407d6e6cff4ad1b2e2a887b163317f]) successfully deleted
I20260809 02:16:24.193234 26955 catalog_manager.cc:4841] TS a1a992abbf44400bbf100e73e082e99f (127.15.75.129:41883): tablet e09e1ef767fd4b57b3a9a464e9029610 (table test-table [id=f0407d6e6cff4ad1b2e2a887b163317f]) successfully deleted
I20260809 02:16:24.193360 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 27095
I20260809 02:16:24.212291 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 27303
I20260809 02:16:24.223188 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 27519
I20260809 02:16:24.233203 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 26929
2026-08-09T02:16:24Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3780 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786241741788589-15662-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:16:24 dist-test-slave-s11t krb5kdc[27943](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:16:24 dist-test-slave-s11t krb5kdc[27943](info): set up 2 sockets
Aug 09 02:16:24 dist-test-slave-s11t krb5kdc[27943](info): commencing operation
krb5kdc: starting...
W20260809 02:16:25.473697 15662 mini_kdc.cc:121] Time spent starting KDC: real 1.211s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Aug 09 02:16:25 dist-test-slave-s11t krb5kdc[27943](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241785, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:16:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:16:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.75.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab.
Entry for principal kudu/127.15.75.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab.
I20260809 02:16:25.593618 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786241741788589-15662-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786241741788589-15662-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786241741788589-15662-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:46329
--webserver_interface=127.15.75.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab
--principal=kudu/127.15.75.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:43021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.75.190:46329
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:16:25.708847 27969 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:16:25.709087 27969 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:16:25.709125 27969 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:16:25.712960 27969 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:16:25.713003 27969 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:16:25.713022 27969 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:16:25.713038 27969 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:16:25.713053 27969 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:16:25.718005 27969 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:43021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786241741788589-15662-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786241741788589-15662-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.75.190:46329
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:46329
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.75.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27969
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786241741788589-15662-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:16:25.718964 27969 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:16:25.719843 27969 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:16:25.728068 27982 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:25.728061 27976 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:25.728113 27969 server_base.cc:1034] running on GCE node
W20260809 02:16:25.728062 27977 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:25.729001 27969 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:16:25.730018 27969 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:16:25.731205 27969 hybrid_clock.cc:648] HybridClock initialized: now 1786241785731183 us; error 46 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-09T02:16:25Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1501 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:16:25 dist-test-slave-s11t krb5kdc[27987](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:16:25 dist-test-slave-s11t krb5kdc[27987](info): set up 2 sockets
Aug 09 02:16:25 dist-test-slave-s11t krb5kdc[27987](info): commencing operation
krb5kdc: starting...
W20260809 02:16:26.971824 15662 mini_kdc.cc:121] Time spent starting KDC: real 1.208s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Aug 09 02:16:27 dist-test-slave-s11t krb5kdc[27987](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241787, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:16:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:16:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.75.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab.
Entry for principal kudu/127.15.75.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab.
I20260809 02:16:27.095345 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:36831
--webserver_interface=127.15.75.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab
--principal=kudu/127.15.75.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:46275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.75.190:36831
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:16:27.205552 28004 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:16:27.205822 28004 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:16:27.205892 28004 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:16:27.209863 28004 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:16:27.209942 28004 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:16:27.209982 28004 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:16:27.210038 28004 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:16:27.210078 28004 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:16:27.215137 28004 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:46275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.75.190:36831
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab
--principal=kudu/127.15.75.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:36831
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.75.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28004
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:16:27.216102 28004 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:16:27.217101 28004 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:16:27.225479 28014 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:27.225490 28013 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:27.225556 28016 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:27.225754 28004 server_base.cc:1034] running on GCE node
I20260809 02:16:27.226482 28004 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:16:27.227447 28004 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:16:27.228602 28004 hybrid_clock.cc:648] HybridClock initialized: now 1786241787228573 us; error 41 us; skew 500 ppm
Aug 09 02:16:27 dist-test-slave-s11t krb5kdc[27987](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241787, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:16:27.231380 28004 init.cc:377] Logged in from keytab as kudu/127.15.75.190@KRBTEST.COM (short username kudu)
I20260809 02:16:27.232285 28004 webserver.cc:460] Webserver started at http://127.15.75.190:41813/ using document root <none> and password file <none>
I20260809 02:16:27.232906 28004 fs_manager.cc:359] Metadata directory not provided
I20260809 02:16:27.232997 28004 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:16:27.233227 28004 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:16:27.235284 28004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/master-0/data/instance:
uuid: "b92dc7ad02594597bd30aa317c5ba2ca"
format_stamp: "Formatted at 2026-08-09 02:16:27 on dist-test-slave-s11t"
server_key: "839293153bec1e0289a506e3e4ad7314"
server_key_iv: "df46fb56482e972ef41f5c7972f24358"
server_key_version: "encryptionkey@0"
I20260809 02:16:27.235857 28004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/master-0/wal/instance:
uuid: "b92dc7ad02594597bd30aa317c5ba2ca"
format_stamp: "Formatted at 2026-08-09 02:16:27 on dist-test-slave-s11t"
server_key: "839293153bec1e0289a506e3e4ad7314"
server_key_iv: "df46fb56482e972ef41f5c7972f24358"
server_key_version: "encryptionkey@0"
I20260809 02:16:27.239464 28004 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260809 02:16:27.241788 28030 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:27.242996 28004 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 02:16:27.243144 28004 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/master-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/master-0/wal
uuid: "b92dc7ad02594597bd30aa317c5ba2ca"
format_stamp: "Formatted at 2026-08-09 02:16:27 on dist-test-slave-s11t"
server_key: "839293153bec1e0289a506e3e4ad7314"
server_key_iv: "df46fb56482e972ef41f5c7972f24358"
server_key_version: "encryptionkey@0"
I20260809 02:16:27.243255 28004 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:16:27.256896 28004 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:16:27.259651 28004 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:16:27.259847 28004 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:16:27.267135 28004 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.190:36831
I20260809 02:16:27.267165 28111 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.190:36831 every 8 connection(s)
I20260809 02:16:27.268026 28004 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
I20260809 02:16:27.270689 28112 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:16:27.271070 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 28004
I20260809 02:16:27.271205 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/master-0/wal/instance
I20260809 02:16:27.271534 15662 external_mini_cluster.cc:1428] Setting key a9b8b93f11c63428a38f2cc9ce87593e
I20260809 02:16:27.277201 28112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b92dc7ad02594597bd30aa317c5ba2ca: Bootstrap starting.
I20260809 02:16:27.279452 28112 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b92dc7ad02594597bd30aa317c5ba2ca: Neither blocks nor log segments found. Creating new log.
Aug 09 02:16:27 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241787, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:16:27.280359 28112 log.cc:824] T 00000000000000000000000000000000 P b92dc7ad02594597bd30aa317c5ba2ca: Log is configured to *not* fsync() on all Append() calls
I20260809 02:16:27.282528 28112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b92dc7ad02594597bd30aa317c5ba2ca: No bootstrap required, opened a new log
I20260809 02:16:27.285604 28118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:27.273008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51746 (local address 127.15.75.190:36831)
0809 02:16:27.273425 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:27.273439 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:27.273472 (+ 33us) server_negotiation.cc:407] Connection header received
0809 02:16:27.274129 (+ 657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:27.274145 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:27.274442 (+ 297us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:27.274763 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:27.275577 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:27.276529 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:27.277510 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:27.277976 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:27.278143 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:27.280951 (+ 2808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:27.280961 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:27.280980 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:27.281011 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:27.282789 (+ 1778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:27.283293 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:27.283297 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:27.283301 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:27.283411 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:27.283714 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:27.283717 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:27.283718 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:27.284115 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:27.284263 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:27.284573 (+ 310us) server_negotiation.cc:299] Negotiation successful
0809 02:16:27.284936 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":120,"threads_started":1}
I20260809 02:16:27.286566 28112 raft_consensus.cc:348] T 00000000000000000000000000000000 P b92dc7ad02594597bd30aa317c5ba2ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b92dc7ad02594597bd30aa317c5ba2ca" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 36831 } }
I20260809 02:16:27.286830 28112 raft_consensus.cc:374] T 00000000000000000000000000000000 P b92dc7ad02594597bd30aa317c5ba2ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:16:27.286900 28112 raft_consensus.cc:729] T 00000000000000000000000000000000 P b92dc7ad02594597bd30aa317c5ba2ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b92dc7ad02594597bd30aa317c5ba2ca, State: Initialized, Role: FOLLOWER
I20260809 02:16:27.287355 28112 consensus_queue.cc:260] T 00000000000000000000000000000000 P b92dc7ad02594597bd30aa317c5ba2ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b92dc7ad02594597bd30aa317c5ba2ca" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 36831 } }
I20260809 02:16:27.287509 28112 raft_consensus.cc:388] T 00000000000000000000000000000000 P b92dc7ad02594597bd30aa317c5ba2ca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:16:27.287576 28112 raft_consensus.cc:482] T 00000000000000000000000000000000 P b92dc7ad02594597bd30aa317c5ba2ca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:16:27.287674 28112 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b92dc7ad02594597bd30aa317c5ba2ca [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:16:27.288606 28112 raft_consensus.cc:504] T 00000000000000000000000000000000 P b92dc7ad02594597bd30aa317c5ba2ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b92dc7ad02594597bd30aa317c5ba2ca" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 36831 } }
I20260809 02:16:27.288954 28112 leader_election.cc:302] T 00000000000000000000000000000000 P b92dc7ad02594597bd30aa317c5ba2ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b92dc7ad02594597bd30aa317c5ba2ca; no voters:
I20260809 02:16:27.289253 28112 leader_election.cc:288] T 00000000000000000000000000000000 P b92dc7ad02594597bd30aa317c5ba2ca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:16:27.289389 28122 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b92dc7ad02594597bd30aa317c5ba2ca [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:16:27.289626 28122 raft_consensus.cc:686] T 00000000000000000000000000000000 P b92dc7ad02594597bd30aa317c5ba2ca [term 1 LEADER]: Becoming Leader. State: Replica: b92dc7ad02594597bd30aa317c5ba2ca, State: Running, Role: LEADER
I20260809 02:16:27.289978 28122 consensus_queue.cc:237] T 00000000000000000000000000000000 P b92dc7ad02594597bd30aa317c5ba2ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b92dc7ad02594597bd30aa317c5ba2ca" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 36831 } }
I20260809 02:16:27.290462 28112 sys_catalog.cc:564] T 00000000000000000000000000000000 P b92dc7ad02594597bd30aa317c5ba2ca [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:16:27.291777 28124 sys_catalog.cc:455] T 00000000000000000000000000000000 P b92dc7ad02594597bd30aa317c5ba2ca [sys.catalog]: SysCatalogTable state changed. Reason: New leader b92dc7ad02594597bd30aa317c5ba2ca. Latest consensus state: current_term: 1 leader_uuid: "b92dc7ad02594597bd30aa317c5ba2ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b92dc7ad02594597bd30aa317c5ba2ca" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 36831 } } }
I20260809 02:16:27.291788 28123 sys_catalog.cc:455] T 00000000000000000000000000000000 P b92dc7ad02594597bd30aa317c5ba2ca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b92dc7ad02594597bd30aa317c5ba2ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b92dc7ad02594597bd30aa317c5ba2ca" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 36831 } } }
I20260809 02:16:27.291910 28124 sys_catalog.cc:458] T 00000000000000000000000000000000 P b92dc7ad02594597bd30aa317c5ba2ca [sys.catalog]: This master's current role is: LEADER
I20260809 02:16:27.291931 28123 sys_catalog.cc:458] T 00000000000000000000000000000000 P b92dc7ad02594597bd30aa317c5ba2ca [sys.catalog]: This master's current role is: LEADER
I20260809 02:16:27.292264 28132 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:16:27.295351 28132 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:16:27.300434 28132 catalog_manager.cc:1347] Generated new cluster ID: 9df18dab07914699945129219ddae6da
I20260809 02:16:27.300499 28132 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:16:27.319689 28132 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:16:27.320618 28132 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:16:27.326262 28132 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b92dc7ad02594597bd30aa317c5ba2ca: Generated new TSK 0
I20260809 02:16:27.326906 28132 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.75.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab.
Entry for principal kudu/127.15.75.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab.
I20260809 02:16:27.361538 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.129:0
--local_ip_for_outbound_sockets=127.15.75.129
--webserver_interface=127.15.75.129
--webserver_port=0
--tserver_master_addrs=127.15.75.190:36831
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab
--principal=kudu/127.15.75.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:46275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:16:27.469903 28149 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:16:27.470168 28149 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:16:27.470242 28149 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:16:27.474236 28149 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:16:27.474314 28149 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:16:27.474403 28149 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.129
I20260809 02:16:27.479439 28149 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:46275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab
--principal=kudu/127.15.75.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.75.129
--webserver_port=0
--tserver_master_addrs=127.15.75.190:36831
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28149
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.129
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:16:27.480407 28149 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:16:27.481423 28149 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:16:27.490806 28163 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:27.490859 28158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:27.490885 28149 server_base.cc:1034] running on GCE node
W20260809 02:16:27.490885 28160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:27.491439 28149 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:16:27.492014 28149 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:16:27.493213 28149 hybrid_clock.cc:648] HybridClock initialized: now 1786241787493199 us; error 27 us; skew 500 ppm
Aug 09 02:16:27 dist-test-slave-s11t krb5kdc[27987](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241787, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:16:27.495869 28149 init.cc:377] Logged in from keytab as kudu/127.15.75.129@KRBTEST.COM (short username kudu)
I20260809 02:16:27.496765 28149 webserver.cc:460] Webserver started at http://127.15.75.129:44989/ using document root <none> and password file <none>
I20260809 02:16:27.497290 28149 fs_manager.cc:359] Metadata directory not provided
I20260809 02:16:27.497351 28149 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:16:27.497561 28149 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:16:27.499809 28149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-0/data/instance:
uuid: "31bcd97fde634f2aba79a37d17839f99"
format_stamp: "Formatted at 2026-08-09 02:16:27 on dist-test-slave-s11t"
server_key: "f23ad169ac839688de9897e99f496358"
server_key_iv: "a99fd83b18d9628da9eedd507a36af39"
server_key_version: "encryptionkey@0"
I20260809 02:16:27.500396 28149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-0/wal/instance:
uuid: "31bcd97fde634f2aba79a37d17839f99"
format_stamp: "Formatted at 2026-08-09 02:16:27 on dist-test-slave-s11t"
server_key: "f23ad169ac839688de9897e99f496358"
server_key_iv: "a99fd83b18d9628da9eedd507a36af39"
server_key_version: "encryptionkey@0"
I20260809 02:16:27.504074 28149 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260809 02:16:27.506490 28178 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:27.507651 28149 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 02:16:27.507788 28149 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-0/wal
uuid: "31bcd97fde634f2aba79a37d17839f99"
format_stamp: "Formatted at 2026-08-09 02:16:27 on dist-test-slave-s11t"
server_key: "f23ad169ac839688de9897e99f496358"
server_key_iv: "a99fd83b18d9628da9eedd507a36af39"
server_key_version: "encryptionkey@0"
I20260809 02:16:27.507889 28149 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:16:27.523404 28149 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:16:27.526152 28149 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:16:27.526335 28149 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:16:27.526888 28149 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:16:27.527678 28149 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:16:27.527724 28149 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:27.527786 28149 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:16:27.527827 28149 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:27.537264 28149 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.129:45879
I20260809 02:16:27.537288 28363 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.129:45879 every 8 connection(s)
I20260809 02:16:27.538100 28149 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
Aug 09 02:16:27 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241787, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.129@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:16:27.548313 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 28149
I20260809 02:16:27.548462 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-0/wal/instance
I20260809 02:16:27.548882 15662 external_mini_cluster.cc:1428] Setting key d810fb4386a9bca2f4b2bdc3b5634972
I20260809 02:16:27.551563 28118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:27.539805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.129:49741 (local address 127.15.75.190:36831)
0809 02:16:27.539945 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:27.539949 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:27.540723 (+ 774us) server_negotiation.cc:407] Connection header received
0809 02:16:27.541973 (+ 1250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:27.541976 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:27.542022 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:27.542095 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:27.543177 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:27.543702 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:27.544868 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:27.545091 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:27.545170 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:27.547655 (+ 2485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:27.547658 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:27.547660 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:27.547682 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:27.549299 (+ 1617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:27.549873 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:27.549876 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:27.549877 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:27.549961 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:27.550378 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:27.550382 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:27.550383 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:27.550609 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:27.550717 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:27.551166 (+ 449us) server_negotiation.cc:299] Negotiation successful
0809 02:16:27.551308 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 02:16:27.552106 28366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:27.540141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36831 (local address 127.15.75.129:49741)
0809 02:16:27.540552 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:27.540592 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:27.541782 (+ 1190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:27.542250 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:27.542257 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:27.542597 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:27.543054 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:27.543068 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:27.543813 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:27.543816 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:27.544756 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:27.544773 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:27.545216 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:27.545218 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:27.545372 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:27.545974 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:27.545998 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:27.547514 (+ 1516us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:16:27.549416 (+ 1902us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:27.549422 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:27.549436 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:27.549712 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:27.550061 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:27.550065 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:27.550067 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:27.550268 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:27.550726 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:16:27.550731 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:16:27.550941 (+ 210us) client_negotiation.cc:769] Sending connection context
0809 02:16:27.551084 (+ 143us) client_negotiation.cc:240] Negotiation successful
0809 02:16:27.551312 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":109,"threads_started":1}
I20260809 02:16:27.553365 28364 heartbeater.cc:347] Connected to a master server at 127.15.75.190:36831
I20260809 02:16:27.553606 28364 heartbeater.cc:464] Registering TS with master...
I20260809 02:16:27.554059 28364 heartbeater.cc:511] Master 127.15.75.190:36831 requested a full tablet report, sending...
I20260809 02:16:27.555516 28064 ts_manager.cc:194] Registered new tserver with Master: 31bcd97fde634f2aba79a37d17839f99 (127.15.75.129:45879)
I20260809 02:16:27.556617 28064 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.75.129@KRBTEST.COM'} at 127.15.75.129:49741
WARNING: no policy specified for kudu/127.15.75.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab.
Entry for principal kudu/127.15.75.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab.
I20260809 02:16:27.575738 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.130:0
--local_ip_for_outbound_sockets=127.15.75.130
--webserver_interface=127.15.75.130
--webserver_port=0
--tserver_master_addrs=127.15.75.190:36831
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab
--principal=kudu/127.15.75.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:46275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:16:27.684170 28371 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:16:27.684432 28371 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:16:27.684505 28371 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:16:27.688365 28371 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:16:27.688444 28371 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:16:27.688530 28371 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.130
I20260809 02:16:27.693713 28371 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:46275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab
--principal=kudu/127.15.75.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.75.130
--webserver_port=0
--tserver_master_addrs=127.15.75.190:36831
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28371
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.130
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:16:27.694629 28371 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:16:27.695561 28371 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 02:16:27.704789 28371 server_base.cc:1034] running on GCE node
W20260809 02:16:27.704758 28381 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:27.704769 28384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:27.704943 28380 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:27.705436 28371 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:16:27.706033 28371 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:16:27.707217 28371 hybrid_clock.cc:648] HybridClock initialized: now 1786241787707205 us; error 31 us; skew 500 ppm
Aug 09 02:16:27 dist-test-slave-s11t krb5kdc[27987](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241787, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:16:27.709879 28371 init.cc:377] Logged in from keytab as kudu/127.15.75.130@KRBTEST.COM (short username kudu)
I20260809 02:16:27.710709 28371 webserver.cc:460] Webserver started at http://127.15.75.130:44925/ using document root <none> and password file <none>
I20260809 02:16:27.711237 28371 fs_manager.cc:359] Metadata directory not provided
I20260809 02:16:27.711328 28371 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:16:27.711593 28371 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:16:27.713553 28371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-1/data/instance:
uuid: "f63a46924e3c43108fd7793f07a72240"
format_stamp: "Formatted at 2026-08-09 02:16:27 on dist-test-slave-s11t"
server_key: "dc1030f6cea3316614ede385e2c9540c"
server_key_iv: "f946f8e5764ff69ff9c65998e35dead5"
server_key_version: "encryptionkey@0"
I20260809 02:16:27.714098 28371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-1/wal/instance:
uuid: "f63a46924e3c43108fd7793f07a72240"
format_stamp: "Formatted at 2026-08-09 02:16:27 on dist-test-slave-s11t"
server_key: "dc1030f6cea3316614ede385e2c9540c"
server_key_iv: "f946f8e5764ff69ff9c65998e35dead5"
server_key_version: "encryptionkey@0"
I20260809 02:16:27.717535 28371 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 02:16:27.719727 28397 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:27.720865 28371 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 02:16:27.720993 28371 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-1/wal
uuid: "f63a46924e3c43108fd7793f07a72240"
format_stamp: "Formatted at 2026-08-09 02:16:27 on dist-test-slave-s11t"
server_key: "dc1030f6cea3316614ede385e2c9540c"
server_key_iv: "f946f8e5764ff69ff9c65998e35dead5"
server_key_version: "encryptionkey@0"
I20260809 02:16:27.721095 28371 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:16:27.742318 28371 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:16:27.745007 28371 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:16:27.745203 28371 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:16:27.745787 28371 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:16:27.746726 28371 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:16:27.746793 28371 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:27.746861 28371 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:16:27.746914 28371 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:27.756098 28371 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.130:35245
I20260809 02:16:27.756141 28582 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.130:35245 every 8 connection(s)
I20260809 02:16:27.756976 28371 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
I20260809 02:16:27.761518 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 28371
I20260809 02:16:27.761667 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-1/wal/instance
I20260809 02:16:27.762060 15662 external_mini_cluster.cc:1428] Setting key f63a1adce4891b4c3ec7c9afc8e37e26
Aug 09 02:16:27 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241787, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.130@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:16:27.771446 28118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:27.758673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.130:37415 (local address 127.15.75.190:36831)
0809 02:16:27.758829 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:27.758834 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:27.759665 (+ 831us) server_negotiation.cc:407] Connection header received
0809 02:16:27.760968 (+ 1303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:27.760971 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:27.761017 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:27.761090 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:27.762721 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:27.763440 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:27.764584 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:27.764868 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:27.764983 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:27.767791 (+ 2808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:27.767794 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:27.767796 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:27.767819 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:27.769243 (+ 1424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:27.769862 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:27.769865 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:27.769866 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:27.769945 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:27.770360 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:27.770363 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:27.770364 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:27.770537 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:27.770610 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:27.771230 (+ 620us) server_negotiation.cc:299] Negotiation successful
0809 02:16:27.771321 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 02:16:27.772557 28588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:27.759090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36831 (local address 127.15.75.130:37415)
0809 02:16:27.759510 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:27.759550 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:27.760774 (+ 1224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:27.761273 (+ 499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:27.761292 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:27.761802 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:27.762536 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:27.762558 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:27.763575 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:27.763583 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:27.764456 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:27.764465 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:27.765019 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:27.765023 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:27.765199 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:27.765808 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:27.765833 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:27.767629 (+ 1796us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:16:27.769355 (+ 1726us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:27.769362 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:27.769378 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:27.769753 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:27.770045 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:27.770051 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:27.770056 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:27.770264 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:27.770645 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:16:27.770654 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:16:27.770974 (+ 320us) client_negotiation.cc:769] Sending connection context
0809 02:16:27.771184 (+ 210us) client_negotiation.cc:240] Negotiation successful
0809 02:16:27.771521 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":113,"threads_started":1}
WARNING: no policy specified for kudu/127.15.75.131@KRBTEST.COM; defaulting to no policy
I20260809 02:16:27.773759 28585 heartbeater.cc:347] Connected to a master server at 127.15.75.190:36831
I20260809 02:16:27.773955 28585 heartbeater.cc:464] Registering TS with master...
I20260809 02:16:27.774400 28585 heartbeater.cc:511] Master 127.15.75.190:36831 requested a full tablet report, sending...
I20260809 02:16:27.775465 28064 ts_manager.cc:194] Registered new tserver with Master: f63a46924e3c43108fd7793f07a72240 (127.15.75.130:35245)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.131@KRBTEST.COM" created.
I20260809 02:16:27.776206 28064 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.75.130@KRBTEST.COM'} at 127.15.75.130:37415
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab.
Entry for principal kudu/127.15.75.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab.
I20260809 02:16:27.791452 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.131:0
--local_ip_for_outbound_sockets=127.15.75.131
--webserver_interface=127.15.75.131
--webserver_port=0
--tserver_master_addrs=127.15.75.190:36831
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab
--principal=kudu/127.15.75.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:46275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:16:27.901777 28594 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:16:27.902009 28594 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:16:27.902048 28594 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:16:27.905949 28594 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:16:27.906000 28594 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:16:27.906054 28594 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.131
I20260809 02:16:27.911197 28594 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:46275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab
--principal=kudu/127.15.75.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.75.131
--webserver_port=0
--tserver_master_addrs=127.15.75.190:36831
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28594
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.131
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:16:27.912232 28594 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:16:27.913220 28594 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:16:27.922586 28602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:27.922571 28605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:27.922788 28594 server_base.cc:1034] running on GCE node
W20260809 02:16:27.922588 28603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:27.923271 28594 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:16:27.923892 28594 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:16:27.925047 28594 hybrid_clock.cc:648] HybridClock initialized: now 1786241787925018 us; error 45 us; skew 500 ppm
Aug 09 02:16:27 dist-test-slave-s11t krb5kdc[27987](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241787, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:16:27.927728 28594 init.cc:377] Logged in from keytab as kudu/127.15.75.131@KRBTEST.COM (short username kudu)
I20260809 02:16:27.928570 28594 webserver.cc:460] Webserver started at http://127.15.75.131:44141/ using document root <none> and password file <none>
I20260809 02:16:27.929142 28594 fs_manager.cc:359] Metadata directory not provided
I20260809 02:16:27.929205 28594 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:16:27.929378 28594 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:16:27.931282 28594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-2/data/instance:
uuid: "caa0be2a61cf4e79a4f1e2a4ffdd8090"
format_stamp: "Formatted at 2026-08-09 02:16:27 on dist-test-slave-s11t"
server_key: "ff21a6837fee2acfc1f17223ec27d302"
server_key_iv: "26402c146eeff62ecc0e9f31c1966f4b"
server_key_version: "encryptionkey@0"
I20260809 02:16:27.931808 28594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-2/wal/instance:
uuid: "caa0be2a61cf4e79a4f1e2a4ffdd8090"
format_stamp: "Formatted at 2026-08-09 02:16:27 on dist-test-slave-s11t"
server_key: "ff21a6837fee2acfc1f17223ec27d302"
server_key_iv: "26402c146eeff62ecc0e9f31c1966f4b"
server_key_version: "encryptionkey@0"
I20260809 02:16:27.935446 28594 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260809 02:16:27.937855 28614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:27.939025 28594 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 02:16:27.939257 28594 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-2/wal
uuid: "caa0be2a61cf4e79a4f1e2a4ffdd8090"
format_stamp: "Formatted at 2026-08-09 02:16:27 on dist-test-slave-s11t"
server_key: "ff21a6837fee2acfc1f17223ec27d302"
server_key_iv: "26402c146eeff62ecc0e9f31c1966f4b"
server_key_version: "encryptionkey@0"
I20260809 02:16:27.939371 28594 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:16:27.949128 28594 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:16:27.952016 28594 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:16:27.952232 28594 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:16:27.952874 28594 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:16:27.953665 28594 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:16:27.953737 28594 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:27.953804 28594 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:16:27.953859 28594 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:27.962852 28594 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.131:44443
I20260809 02:16:27.962881 28785 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.131:44443 every 8 connection(s)
I20260809 02:16:27.963887 28594 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
I20260809 02:16:27.967360 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 28594
I20260809 02:16:27.967509 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-2/wal/instance
I20260809 02:16:27.967815 15662 external_mini_cluster.cc:1428] Setting key d50b8ca955c400e5ebdb5809c60df928
Aug 09 02:16:27 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241787, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.131@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:16:27.978008 28118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:27.965677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.131:44985 (local address 127.15.75.190:36831)
0809 02:16:27.965829 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:27.965835 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:27.966682 (+ 847us) server_negotiation.cc:407] Connection header received
0809 02:16:27.968175 (+ 1493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:27.968179 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:27.968240 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:27.968339 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:27.969852 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:27.970447 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:27.971615 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:27.971846 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:27.971945 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:27.974501 (+ 2556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:27.974504 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:27.974506 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:27.974530 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:27.975865 (+ 1335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:27.976383 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:27.976386 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:27.976390 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:27.976470 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:27.976909 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:27.976912 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:27.976913 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:27.977100 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:27.977208 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:27.977707 (+ 499us) server_negotiation.cc:299] Negotiation successful
0809 02:16:27.977832 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 02:16:27.978758 28788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:27.966100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36831 (local address 127.15.75.131:44985)
0809 02:16:27.966533 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:27.966575 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:27.967969 (+ 1394us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:27.968517 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:27.968526 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:27.969031 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:27.969683 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:27.969701 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:27.970568 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:27.970571 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:27.971469 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:27.971478 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:27.972011 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:27.972014 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:27.972171 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:27.972835 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:27.972852 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:27.974322 (+ 1470us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:16:27.975988 (+ 1666us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:27.975994 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:27.976008 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:27.976267 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:27.976576 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:27.976579 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:27.976581 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:27.976802 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:27.977219 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:16:27.977228 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:16:27.977469 (+ 241us) client_negotiation.cc:769] Sending connection context
0809 02:16:27.977648 (+ 179us) client_negotiation.cc:240] Negotiation successful
0809 02:16:27.977973 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":111,"threads_started":1}
I20260809 02:16:27.979905 28786 heartbeater.cc:347] Connected to a master server at 127.15.75.190:36831
I20260809 02:16:27.980167 28786 heartbeater.cc:464] Registering TS with master...
I20260809 02:16:27.980734 28786 heartbeater.cc:511] Master 127.15.75.190:36831 requested a full tablet report, sending...
I20260809 02:16:27.981773 28064 ts_manager.cc:194] Registered new tserver with Master: caa0be2a61cf4e79a4f1e2a4ffdd8090 (127.15.75.131:44443)
I20260809 02:16:27.982371 28064 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.75.131@KRBTEST.COM'} at 127.15.75.131:44985
I20260809 02:16:27.993592 15662 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.15.75.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.75.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.75.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/krb5kdc/oryx_127.15.75.129.keytab.
Entry for principal oryx/127.15.75.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/krb5kdc/oryx_127.15.75.129.keytab.
WARNING: no policy specified for oryx/127.15.75.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.75.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.75.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/krb5kdc/oryx_127.15.75.130.keytab.
Entry for principal oryx/127.15.75.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/krb5kdc/oryx_127.15.75.130.keytab.
WARNING: no policy specified for oryx/127.15.75.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.75.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.75.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/krb5kdc/oryx_127.15.75.131.keytab.
Entry for principal oryx/127.15.75.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/krb5kdc/oryx_127.15.75.131.keytab.
I20260809 02:16:28.066963 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 28149
I20260809 02:16:28.079880 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 28371
I20260809 02:16:28.088485 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 28594
I20260809 02:16:28.097605 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 28004
I20260809 02:16:28.109586 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:36831
--webserver_interface=127.15.75.190
--webserver_port=41813
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab
--principal=kudu/127.15.75.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:46275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.75.190:36831
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:16:28.220662 28798 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:16:28.220924 28798 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:16:28.220963 28798 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:16:28.225551 28798 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:16:28.225602 28798 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:16:28.225622 28798 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:16:28.225638 28798 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:16:28.225685 28798 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:16:28.230777 28798 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:46275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.75.190:36831
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab
--principal=kudu/127.15.75.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:36831
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.75.190
--webserver_port=41813
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28798
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:16:28.231745 28798 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:16:28.232790 28798 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:16:28.241029 28806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:28.241051 28808 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:28.241178 28798 server_base.cc:1034] running on GCE node
W20260809 02:16:28.241391 28805 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:28.241966 28798 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:16:28.242965 28798 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:16:28.244130 28798 hybrid_clock.cc:648] HybridClock initialized: now 1786241788244112 us; error 29 us; skew 500 ppm
Aug 09 02:16:28 dist-test-slave-s11t krb5kdc[27987](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241788, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:16:28.247259 28798 init.cc:377] Logged in from keytab as kudu/127.15.75.190@KRBTEST.COM (short username kudu)
I20260809 02:16:28.248195 28798 webserver.cc:460] Webserver started at http://127.15.75.190:41813/ using document root <none> and password file <none>
I20260809 02:16:28.248803 28798 fs_manager.cc:359] Metadata directory not provided
I20260809 02:16:28.248917 28798 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:16:28.254002 28798 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 02:16:28.256409 28821 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:28.257633 28798 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 02:16:28.257761 28798 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/master-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/master-0/wal
uuid: "b92dc7ad02594597bd30aa317c5ba2ca"
format_stamp: "Formatted at 2026-08-09 02:16:27 on dist-test-slave-s11t"
server_key: "839293153bec1e0289a506e3e4ad7314"
server_key_iv: "df46fb56482e972ef41f5c7972f24358"
server_key_version: "encryptionkey@0"
I20260809 02:16:28.258271 28798 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:16:28.282256 28798 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:16:28.285123 28798 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:16:28.285326 28798 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:16:28.292309 28798 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.190:36831
I20260809 02:16:28.292368 28900 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.190:36831 every 8 connection(s)
I20260809 02:16:28.293447 28798 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
I20260809 02:16:28.295843 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 28798
I20260809 02:16:28.296311 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.129:45879
--local_ip_for_outbound_sockets=127.15.75.129
--tserver_master_addrs=127.15.75.190:36831
--webserver_port=44989
--webserver_interface=127.15.75.129
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab
--principal=kudu/127.15.75.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:46275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.15.75.129
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/krb5kdc/oryx_127.15.75.129.keytab with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 02:16:28.297820 28901 sys_catalog.cc:263] Verifying existing consensus state
I20260809 02:16:28.300101 28901 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b92dc7ad02594597bd30aa317c5ba2ca: Bootstrap starting.
I20260809 02:16:28.317127 28901 log.cc:824] T 00000000000000000000000000000000 P b92dc7ad02594597bd30aa317c5ba2ca: Log is configured to *not* fsync() on all Append() calls
I20260809 02:16:28.323469 28901 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b92dc7ad02594597bd30aa317c5ba2ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 02:16:28.323848 28901 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b92dc7ad02594597bd30aa317c5ba2ca: Bootstrap complete.
I20260809 02:16:28.328085 28901 raft_consensus.cc:348] T 00000000000000000000000000000000 P b92dc7ad02594597bd30aa317c5ba2ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b92dc7ad02594597bd30aa317c5ba2ca" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 36831 } }
I20260809 02:16:28.328460 28901 raft_consensus.cc:729] T 00000000000000000000000000000000 P b92dc7ad02594597bd30aa317c5ba2ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b92dc7ad02594597bd30aa317c5ba2ca, State: Initialized, Role: FOLLOWER
I20260809 02:16:28.329002 28901 consensus_queue.cc:260] T 00000000000000000000000000000000 P b92dc7ad02594597bd30aa317c5ba2ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b92dc7ad02594597bd30aa317c5ba2ca" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 36831 } }
I20260809 02:16:28.329116 28901 raft_consensus.cc:388] T 00000000000000000000000000000000 P b92dc7ad02594597bd30aa317c5ba2ca [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:16:28.329202 28901 raft_consensus.cc:482] T 00000000000000000000000000000000 P b92dc7ad02594597bd30aa317c5ba2ca [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:16:28.329324 28901 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b92dc7ad02594597bd30aa317c5ba2ca [term 1 FOLLOWER]: Advancing to term 2
I20260809 02:16:28.331120 28901 raft_consensus.cc:504] T 00000000000000000000000000000000 P b92dc7ad02594597bd30aa317c5ba2ca [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b92dc7ad02594597bd30aa317c5ba2ca" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 36831 } }
I20260809 02:16:28.331439 28901 leader_election.cc:302] T 00000000000000000000000000000000 P b92dc7ad02594597bd30aa317c5ba2ca [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b92dc7ad02594597bd30aa317c5ba2ca; no voters:
I20260809 02:16:28.331789 28901 leader_election.cc:288] T 00000000000000000000000000000000 P b92dc7ad02594597bd30aa317c5ba2ca [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 02:16:28.331871 28906 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b92dc7ad02594597bd30aa317c5ba2ca [term 2 FOLLOWER]: Leader election won for term 2
I20260809 02:16:28.332087 28906 raft_consensus.cc:686] T 00000000000000000000000000000000 P b92dc7ad02594597bd30aa317c5ba2ca [term 2 LEADER]: Becoming Leader. State: Replica: b92dc7ad02594597bd30aa317c5ba2ca, State: Running, Role: LEADER
I20260809 02:16:28.332372 28906 consensus_queue.cc:237] T 00000000000000000000000000000000 P b92dc7ad02594597bd30aa317c5ba2ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b92dc7ad02594597bd30aa317c5ba2ca" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 36831 } }
I20260809 02:16:28.333035 28901 sys_catalog.cc:564] T 00000000000000000000000000000000 P b92dc7ad02594597bd30aa317c5ba2ca [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:16:28.333299 28906 sys_catalog.cc:455] T 00000000000000000000000000000000 P b92dc7ad02594597bd30aa317c5ba2ca [sys.catalog]: SysCatalogTable state changed. Reason: New leader b92dc7ad02594597bd30aa317c5ba2ca. Latest consensus state: current_term: 2 leader_uuid: "b92dc7ad02594597bd30aa317c5ba2ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b92dc7ad02594597bd30aa317c5ba2ca" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 36831 } } }
I20260809 02:16:28.333431 28906 sys_catalog.cc:458] T 00000000000000000000000000000000 P b92dc7ad02594597bd30aa317c5ba2ca [sys.catalog]: This master's current role is: LEADER
I20260809 02:16:28.333829 28910 sys_catalog.cc:455] T 00000000000000000000000000000000 P b92dc7ad02594597bd30aa317c5ba2ca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "b92dc7ad02594597bd30aa317c5ba2ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b92dc7ad02594597bd30aa317c5ba2ca" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 36831 } } }
I20260809 02:16:28.333988 28910 sys_catalog.cc:458] T 00000000000000000000000000000000 P b92dc7ad02594597bd30aa317c5ba2ca [sys.catalog]: This master's current role is: LEADER
I20260809 02:16:28.335007 28915 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:16:28.338550 28915 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:16:28.339176 28915 catalog_manager.cc:1259] Loaded cluster ID: 9df18dab07914699945129219ddae6da
I20260809 02:16:28.339238 28915 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:16:28.341599 28915 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:16:28.342247 28915 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b92dc7ad02594597bd30aa317c5ba2ca: Loaded TSK: 0
I20260809 02:16:28.343621 28915 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 02:16:28.412376 28904 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:16:28.412652 28904 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:16:28.412768 28904 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:16:28.416759 28904 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:16:28.416828 28904 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:16:28.416913 28904 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.129
I20260809 02:16:28.422019 28904 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:46275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/krb5kdc/oryx_127.15.75.129.keytab
--principal=oryx/127.15.75.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.129:45879
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.75.129
--webserver_port=44989
--tserver_master_addrs=127.15.75.190:36831
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28904
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.129
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:16:28.422992 28904 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:16:28.423969 28904 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:16:28.434751 28952 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:28.434808 28954 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:28.434804 28950 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:28.435454 28904 server_base.cc:1034] running on GCE node
I20260809 02:16:28.435842 28904 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:16:28.436461 28904 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:16:28.437647 28904 hybrid_clock.cc:648] HybridClock initialized: now 1786241788437627 us; error 62 us; skew 500 ppm
Aug 09 02:16:28 dist-test-slave-s11t krb5kdc[27987](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241788, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.75.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:16:28.440567 28904 init.cc:377] Logged in from keytab as oryx/127.15.75.129@KRBTEST.COM (short username oryx)
I20260809 02:16:28.441586 28904 webserver.cc:460] Webserver started at http://127.15.75.129:44989/ using document root <none> and password file <none>
I20260809 02:16:28.442131 28904 fs_manager.cc:359] Metadata directory not provided
I20260809 02:16:28.442236 28904 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:16:28.446107 28904 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 02:16:28.448205 28964 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:28.449515 28904 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 02:16:28.449610 28904 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-0/wal
uuid: "31bcd97fde634f2aba79a37d17839f99"
format_stamp: "Formatted at 2026-08-09 02:16:27 on dist-test-slave-s11t"
server_key: "f23ad169ac839688de9897e99f496358"
server_key_iv: "a99fd83b18d9628da9eedd507a36af39"
server_key_version: "encryptionkey@0"
I20260809 02:16:28.450237 28904 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:16:28.465631 28904 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:16:28.469064 28904 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:16:28.469272 28904 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:16:28.469879 28904 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:16:28.470705 28904 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:16:28.470752 28904 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:28.470865 28904 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:16:28.470898 28904 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:28.479480 28904 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.129:45879
I20260809 02:16:28.479586 29148 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.129:45879 every 8 connection(s)
I20260809 02:16:28.480526 28904 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
I20260809 02:16:28.483901 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 28904
I20260809 02:16:28.484349 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.130:35245
--local_ip_for_outbound_sockets=127.15.75.130
--tserver_master_addrs=127.15.75.190:36831
--webserver_port=44925
--webserver_interface=127.15.75.130
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab
--principal=kudu/127.15.75.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:46275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.15.75.130
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/krb5kdc/oryx_127.15.75.130.keytab with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 02:16:28 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.129@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:16:28 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.129@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
W20260809 02:16:28.495450 29152 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:16:28.482564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.129:55331 (local address 127.15.75.190:36831)
0809 02:16:28.483005 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:28.483022 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:28.483305 (+ 283us) server_negotiation.cc:407] Connection header received
0809 02:16:28.484985 (+ 1680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:28.485004 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:28.485387 (+ 383us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:28.485744 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:28.486873 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:28.488256 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:28.490169 (+ 1913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:28.490621 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:28.490744 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:28.494782 (+ 4038us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.75.129:55331: BlockingRecv error: recv got EOF from 127.15.75.129:55331 (error 108)
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":130,"threads_started":1}
I20260809 02:16:28.495458 29153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:28.482648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36831 (local address 127.15.75.129:55331)
0809 02:16:28.483137 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:28.483186 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:28.484519 (+ 1333us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:28.485900 (+ 1381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:28.485906 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:28.486261 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:28.486686 (+ 425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:28.486698 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:28.488406 (+ 1708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:28.488412 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:28.490034 (+ 1622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:28.490051 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:28.490794 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:28.490801 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:28.491034 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:28.491678 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:28.491704 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:28.494493 (+ 2789us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.75.190:36831: Server oryx/127.15.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":161,"threads_started":1}
W20260809 02:16:28.495891 29150 heartbeater.cc:644] Failed to heartbeat to 127.15.75.190:36831 (0 consecutive failures): Not authorized: Failed to ping master at 127.15.75.190:36831: Client connection negotiation failed: client connection to 127.15.75.190:36831: Server oryx/127.15.75.190@KRBTEST.COM not found in Kerberos database
Aug 09 02:16:28 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.129@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:16:28 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.129@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:16:28.501534 29153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:28.496388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36831 (local address 127.15.75.129:57783)
0809 02:16:28.496503 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:28.496520 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:28.496600 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:28.496878 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:28.496881 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:28.497036 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:28.497120 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:28.497125 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:28.497834 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:28.497839 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:28.498369 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:28.498377 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:28.498953 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:28.498957 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:28.499065 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:28.499545 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:28.499556 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:28.501413 (+ 1857us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.75.190:36831: Server oryx/127.15.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":36}
W20260809 02:16:28.501880 29152 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:16:28.496468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.129:57783 (local address 127.15.75.190:36831)
0809 02:16:28.496577 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:28.496581 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:28.496593 (+ 12us) server_negotiation.cc:407] Connection header received
0809 02:16:28.496729 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:28.496732 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:28.496779 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:28.496863 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:28.497233 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:28.497724 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:28.498492 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:28.498789 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:28.498954 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:28.501546 (+ 2592us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.75.129:57783: BlockingRecv error: recv got EOF from 127.15.75.129:57783 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 09 02:16:28 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.129@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:16:28 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.129@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:16:28.507236 29153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:28.502015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36831 (local address 127.15.75.129:54117)
0809 02:16:28.502244 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:28.502257 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:28.502325 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:28.502532 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:28.502535 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:28.502708 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:28.502797 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:28.502803 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:28.503479 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:28.503482 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:28.504333 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:28.504343 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:28.504815 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:28.504818 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:28.504903 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:28.505307 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:28.505322 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:28.507103 (+ 1781us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.75.190:36831: Server oryx/127.15.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153}
W20260809 02:16:28.507318 29152 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:16:28.502226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.129:54117 (local address 127.15.75.190:36831)
0809 02:16:28.502340 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:28.502343 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:28.502356 (+ 13us) server_negotiation.cc:407] Connection header received
0809 02:16:28.502393 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:28.502395 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:28.502428 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:28.502492 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:28.502908 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:28.503363 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:28.504446 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:28.504643 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:28.504790 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:28.507211 (+ 2421us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.75.129:54117: BlockingRecv error: recv got EOF from 127.15.75.129:54117 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260809 02:16:28.507632 29150 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 02:16:28.602355 29154 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:16:28.602626 29154 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:16:28.602711 29154 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:16:28.606761 29154 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:16:28.606841 29154 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:16:28.606930 29154 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.130
I20260809 02:16:28.611999 29154 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:46275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/krb5kdc/oryx_127.15.75.130.keytab
--principal=oryx/127.15.75.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.130:35245
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.75.130
--webserver_port=44925
--tserver_master_addrs=127.15.75.190:36831
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29154
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.130
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:16:28.613029 29154 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:16:28.614017 29154 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:16:28.623445 29166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:28.623461 29163 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:28.623461 29162 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:28.623685 29154 server_base.cc:1034] running on GCE node
I20260809 02:16:28.624145 29154 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:16:28.624850 29154 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:16:28.626039 29154 hybrid_clock.cc:648] HybridClock initialized: now 1786241788626024 us; error 23 us; skew 500 ppm
Aug 09 02:16:28 dist-test-slave-s11t krb5kdc[27987](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241788, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.75.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:16:28.628747 29154 init.cc:377] Logged in from keytab as oryx/127.15.75.130@KRBTEST.COM (short username oryx)
I20260809 02:16:28.629771 29154 webserver.cc:460] Webserver started at http://127.15.75.130:44925/ using document root <none> and password file <none>
I20260809 02:16:28.630359 29154 fs_manager.cc:359] Metadata directory not provided
I20260809 02:16:28.630437 29154 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:16:28.634223 29154 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 02:16:28.636252 29175 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:28.637480 29154 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 02:16:28.637629 29154 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-1/wal
uuid: "f63a46924e3c43108fd7793f07a72240"
format_stamp: "Formatted at 2026-08-09 02:16:27 on dist-test-slave-s11t"
server_key: "dc1030f6cea3316614ede385e2c9540c"
server_key_iv: "f946f8e5764ff69ff9c65998e35dead5"
server_key_version: "encryptionkey@0"
I20260809 02:16:28.638165 29154 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:16:28.661299 29154 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:16:28.664124 29154 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:16:28.664327 29154 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:16:28.664940 29154 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:16:28.665748 29154 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:16:28.665838 29154 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:28.665915 29154 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:16:28.665969 29154 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:28.674893 29154 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.130:35245
I20260809 02:16:28.674930 29343 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.130:35245 every 8 connection(s)
I20260809 02:16:28.675853 29154 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
I20260809 02:16:28.682364 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 29154
I20260809 02:16:28.682886 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.131:44443
--local_ip_for_outbound_sockets=127.15.75.131
--tserver_master_addrs=127.15.75.190:36831
--webserver_port=44141
--webserver_interface=127.15.75.131
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab
--principal=kudu/127.15.75.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:46275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.15.75.131
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/krb5kdc/oryx_127.15.75.131.keytab with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 02:16:28 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.130@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:16:28 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.130@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
W20260809 02:16:28.687665 29152 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:16:28.677668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.130:49757 (local address 127.15.75.190:36831)
0809 02:16:28.677808 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:28.677812 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:28.678613 (+ 801us) server_negotiation.cc:407] Connection header received
0809 02:16:28.679941 (+ 1328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:28.679945 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:28.680006 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:28.680084 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:28.681212 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:28.681755 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:28.683430 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:28.683655 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:28.683743 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:28.687482 (+ 3739us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.75.130:49757: BlockingRecv error: recv got EOF from 127.15.75.130:49757 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 02:16:28.688045 29349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:28.678023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36831 (local address 127.15.75.130:49757)
0809 02:16:28.678467 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:28.678507 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:28.679725 (+ 1218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:28.680250 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:28.680257 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:28.680587 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:28.681065 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:28.681078 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:28.681899 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:28.681902 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:28.683291 (+ 1389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:28.683305 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:28.683814 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:28.683839 (+ 25us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:28.684065 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:28.684661 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:28.684711 (+ 50us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:28.687283 (+ 2572us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.75.190:36831: Server oryx/127.15.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":129,"threads_started":1}
W20260809 02:16:28.688640 29345 heartbeater.cc:644] Failed to heartbeat to 127.15.75.190:36831 (0 consecutive failures): Not authorized: Failed to ping master at 127.15.75.190:36831: Client connection negotiation failed: client connection to 127.15.75.190:36831: Server oryx/127.15.75.190@KRBTEST.COM not found in Kerberos database
Aug 09 02:16:28 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.130@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:16:28 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.130@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:16:28.694625 29349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:28.689137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36831 (local address 127.15.75.130:60291)
0809 02:16:28.689247 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:28.689262 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:28.689338 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:28.689624 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:28.689626 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:28.689746 (+ 120us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:28.689836 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:28.689840 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:28.690899 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:28.690901 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:28.691410 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:28.691416 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:28.692058 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:28.692061 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:28.692129 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:28.692656 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:28.692667 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:28.694541 (+ 1874us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.75.190:36831: Server oryx/127.15.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":47}
W20260809 02:16:28.694870 29152 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:16:28.689232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.130:60291 (local address 127.15.75.190:36831)
0809 02:16:28.689370 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:28.689376 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:28.689391 (+ 15us) server_negotiation.cc:407] Connection header received
0809 02:16:28.689450 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:28.689455 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:28.689511 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:28.689607 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:28.689948 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:28.690790 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:28.691526 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:28.691933 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:28.692061 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:28.694723 (+ 2662us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.75.130:60291: BlockingRecv error: recv got EOF from 127.15.75.130:60291 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 09 02:16:28 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.130@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:16:28 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.130@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:16:28.700176 29349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:28.695178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36831 (local address 127.15.75.130:41625)
0809 02:16:28.695291 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:28.695304 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:28.695372 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:28.695608 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:28.695610 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:28.695726 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:28.695789 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:28.695793 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:28.696886 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:28.696889 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:28.697382 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:28.697390 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:28.697928 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:28.697930 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:28.697985 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:28.698447 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:28.698459 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:28.700096 (+ 1637us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.75.190:36831: Server oryx/127.15.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":42}
W20260809 02:16:28.700333 29152 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:16:28.695253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.130:41625 (local address 127.15.75.190:36831)
0809 02:16:28.695373 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:28.695378 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:28.695405 (+ 27us) server_negotiation.cc:407] Connection header received
0809 02:16:28.695446 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:28.695451 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:28.695497 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:28.695590 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:28.695923 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:28.696778 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:28.697488 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:28.697823 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:28.697922 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:28.700236 (+ 2314us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.75.130:41625: BlockingRecv error: recv got EOF from 127.15.75.130:41625 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260809 02:16:28.700460 29345 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 02:16:28.797628 29350 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:16:28.797858 29350 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:16:28.797905 29350 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:16:28.801909 29350 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:16:28.801961 29350 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:16:28.802016 29350 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.131
I20260809 02:16:28.807003 29350 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:46275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/krb5kdc/oryx_127.15.75.131.keytab
--principal=oryx/127.15.75.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.131:44443
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.75.131
--webserver_port=44141
--tserver_master_addrs=127.15.75.190:36831
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29350
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.131
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:16:28.807955 29350 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:16:28.808943 29350 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:16:28.818328 29358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:28.818367 29350 server_base.cc:1034] running on GCE node
W20260809 02:16:28.818360 29357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:28.818521 29360 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:28.818888 29350 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:16:28.819578 29350 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:16:28.820875 29350 hybrid_clock.cc:648] HybridClock initialized: now 1786241788820805 us; error 78 us; skew 500 ppm
Aug 09 02:16:28 dist-test-slave-s11t krb5kdc[27987](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241788, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.75.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:16:28.823772 29350 init.cc:377] Logged in from keytab as oryx/127.15.75.131@KRBTEST.COM (short username oryx)
I20260809 02:16:28.824661 29350 webserver.cc:460] Webserver started at http://127.15.75.131:44141/ using document root <none> and password file <none>
I20260809 02:16:28.825273 29350 fs_manager.cc:359] Metadata directory not provided
I20260809 02:16:28.825348 29350 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:16:28.829078 29350 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260809 02:16:28.831282 29370 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:28.832456 29350 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 02:16:28.832583 29350 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-2/wal
uuid: "caa0be2a61cf4e79a4f1e2a4ffdd8090"
format_stamp: "Formatted at 2026-08-09 02:16:27 on dist-test-slave-s11t"
server_key: "ff21a6837fee2acfc1f17223ec27d302"
server_key_iv: "26402c146eeff62ecc0e9f31c1966f4b"
server_key_version: "encryptionkey@0"
I20260809 02:16:28.833225 29350 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:16:28.869421 29350 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:16:28.872341 29350 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:16:28.872560 29350 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:16:28.873202 29350 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:16:28.874047 29350 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:16:28.874095 29350 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:28.874161 29350 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:16:28.874194 29350 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:28.883656 29350 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.131:44443
I20260809 02:16:28.883678 29554 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.131:44443 every 8 connection(s)
I20260809 02:16:28.884636 29350 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
I20260809 02:16:28.891111 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 29350
Aug 09 02:16:28 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.131@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:16:28 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.131@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
W20260809 02:16:28.896034 29152 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:16:28.886403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.131:37787 (local address 127.15.75.190:36831)
0809 02:16:28.886529 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:28.886533 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:28.887308 (+ 775us) server_negotiation.cc:407] Connection header received
0809 02:16:28.888545 (+ 1237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:28.888549 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:28.888599 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:28.888735 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:28.889822 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:28.890355 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:28.891798 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:28.892118 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:28.892244 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:28.895903 (+ 3659us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.75.131:37787: BlockingRecv error: recv got EOF from 127.15.75.131:37787 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 02:16:28.896651 29560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:28.886721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36831 (local address 127.15.75.131:37787)
0809 02:16:28.887144 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:28.887184 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:28.888315 (+ 1131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:28.888888 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:28.888895 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:28.889223 (+ 328us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:28.889675 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:28.889688 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:28.890464 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:28.890466 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:28.891651 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:28.891664 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:28.892286 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:28.892293 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:28.892482 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:28.893105 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:28.893130 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:28.895762 (+ 2632us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.75.190:36831: Server oryx/127.15.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":117,"threads_started":1}
W20260809 02:16:28.897257 29555 heartbeater.cc:644] Failed to heartbeat to 127.15.75.190:36831 (0 consecutive failures): Not authorized: Failed to ping master at 127.15.75.190:36831: Client connection negotiation failed: client connection to 127.15.75.190:36831: Server oryx/127.15.75.190@KRBTEST.COM not found in Kerberos database
Aug 09 02:16:28 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.131@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:16:28.901475 29562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:28.892118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51752 (local address 127.15.75.190:36831)
0809 02:16:28.892373 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:28.892376 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:28.892385 (+ 9us) server_negotiation.cc:407] Connection header received
0809 02:16:28.892412 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:28.892414 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:28.892444 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:28.892512 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:28.893495 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:28.893988 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:28.895364 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:28.895714 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:28.895820 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:28.897513 (+ 1693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:28.897520 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:28.897537 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:28.897570 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:28.899340 (+ 1770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:28.899798 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:28.899811 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:28.899815 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:28.899931 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:28.900221 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:28.900226 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:28.900227 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:28.900546 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:28.900783 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:28.901077 (+ 294us) server_negotiation.cc:299] Negotiation successful
0809 02:16:28.901305 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":86,"threads_started":1}
Aug 09 02:16:28 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.131@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:16:28.902547 29560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:28.897714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36831 (local address 127.15.75.131:38011)
0809 02:16:28.897818 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:28.897835 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:28.897917 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:28.898137 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:28.898140 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:28.898291 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:28.898372 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:28.898377 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:28.899054 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:28.899059 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:28.899577 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:28.899585 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:28.900017 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:28.900021 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:28.900116 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:28.900555 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:28.900566 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:28.902420 (+ 1854us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.75.190:36831: Server oryx/127.15.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":32}
W20260809 02:16:28.902633 29152 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:16:28.897782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.131:38011 (local address 127.15.75.190:36831)
0809 02:16:28.897888 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:28.897892 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:28.897905 (+ 13us) server_negotiation.cc:407] Connection header received
0809 02:16:28.897995 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:28.897998 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:28.898040 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:28.898121 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:28.898483 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:28.898948 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:28.899695 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:28.899910 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:28.900013 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:28.902533 (+ 2520us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.75.131:38011: BlockingRecv error: recv got EOF from 127.15.75.131:38011 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 09 02:16:28 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.131@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:16:28 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.131@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:16:28.907972 29560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:28.903571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36831 (local address 127.15.75.131:51471)
0809 02:16:28.903677 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:28.903690 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:28.903759 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:28.904003 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:28.904005 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:28.904168 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:28.904265 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:28.904271 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:28.904999 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:28.905001 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:28.905511 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:28.905519 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:28.905900 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:28.905903 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:28.905961 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:28.906308 (+ 347us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:28.906319 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:28.907892 (+ 1573us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.75.190:36831: Server oryx/127.15.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":32}
W20260809 02:16:28.908164 29152 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:16:28.903613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.131:51471 (local address 127.15.75.190:36831)
0809 02:16:28.903830 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:28.903833 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:28.903842 (+ 9us) server_negotiation.cc:407] Connection header received
0809 02:16:28.903865 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:28.903867 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:28.903902 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:28.903961 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:28.904358 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:28.904868 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:28.905613 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:28.905804 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:28.905874 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:28.908037 (+ 2163us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.75.131:51471: BlockingRecv error: recv got EOF from 127.15.75.131:51471 (error 108)
Metrics: {"server-negotiator.queue_time_us":170}
W20260809 02:16:28.908236 29555 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 09 02:16:29 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.129@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:16:29 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.129@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:16:29.516376 29563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:29.508184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36831 (local address 127.15.75.129:48247)
0809 02:16:29.508458 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:29.508473 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:29.508558 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:29.508806 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:29.508808 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:29.508994 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:29.509115 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:29.509120 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:29.509844 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:29.509846 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:29.510434 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:29.510443 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:29.510858 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:29.510860 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:29.510936 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:29.514309 (+ 3373us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:29.514321 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:29.516262 (+ 1941us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.75.190:36831: Server oryx/127.15.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":102,"threads_started":1}
W20260809 02:16:29.516587 29564 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:16:29.508258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.129:48247 (local address 127.15.75.190:36831)
0809 02:16:29.508517 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:29.508520 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:29.508533 (+ 13us) server_negotiation.cc:407] Connection header received
0809 02:16:29.508627 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:29.508629 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:29.508670 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:29.508762 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:29.509219 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:29.509746 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:29.510538 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:29.510758 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:29.510832 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:29.516409 (+ 5577us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.75.129:48247: BlockingRecv error: recv got EOF from 127.15.75.129:48247 (error 108)
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":113,"threads_started":1}
Aug 09 02:16:29 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.130@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:16:29 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.130@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:16:29.706421 29565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:29.701102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36831 (local address 127.15.75.130:53135)
0809 02:16:29.701382 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:29.701396 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:29.701492 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:29.701716 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:29.701718 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:29.701907 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:29.702004 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:29.702009 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:29.702723 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:29.702725 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:29.703288 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:29.703295 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:29.703691 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:29.703694 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:29.703754 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:29.704319 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:29.704331 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:29.706214 (+ 1883us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.75.190:36831: Server oryx/127.15.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":107,"threads_started":1}
W20260809 02:16:29.706549 29564 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:16:29.701154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.130:53135 (local address 127.15.75.190:36831)
0809 02:16:29.701298 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:29.701302 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:29.701462 (+ 160us) server_negotiation.cc:407] Connection header received
0809 02:16:29.701567 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:29.701569 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:29.701613 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:29.701679 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:29.702096 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:29.702621 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:29.703383 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:29.703594 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:29.703671 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:29.706418 (+ 2747us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.75.130:53135: BlockingRecv error: recv got EOF from 127.15.75.130:53135 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 09 02:16:29 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.131@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:16:29 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.131@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:16:29.914471 29566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:29.908905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36831 (local address 127.15.75.131:48735)
0809 02:16:29.909189 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:29.909202 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:29.909295 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:29.909525 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:29.909527 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:29.909705 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:29.909800 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:29.909805 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:29.910512 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:29.910515 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:29.911091 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:29.911099 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:29.911490 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:29.911493 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:29.911555 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:29.912187 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:29.912200 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:29.914352 (+ 2152us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.75.190:36831: Server oryx/127.15.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":108,"threads_started":1}
W20260809 02:16:29.914736 29564 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:16:29.908949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.131:48735 (local address 127.15.75.190:36831)
0809 02:16:29.909099 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:29.909103 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:29.909266 (+ 163us) server_negotiation.cc:407] Connection header received
0809 02:16:29.909371 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:29.909374 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:29.909419 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:29.909486 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:29.909890 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:29.910412 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:29.911185 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:29.911397 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:29.911470 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:29.914497 (+ 3027us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.75.131:48735: BlockingRecv error: recv got EOF from 127.15.75.131:48735 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 09 02:16:30 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.129@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:16:30 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.129@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:16:30.522584 29567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:30.517150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36831 (local address 127.15.75.129:33385)
0809 02:16:30.517455 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:30.517468 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:30.517593 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:30.517923 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:30.517926 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:30.518173 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:30.518314 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:30.518323 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:30.519145 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:30.519148 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:30.519711 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:30.519718 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:30.520121 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:30.520123 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:30.520187 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:30.520654 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:30.520665 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:30.522474 (+ 1809us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.75.190:36831: Server oryx/127.15.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":76,"threads_started":1}
W20260809 02:16:30.522768 29568 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:16:30.517229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.129:33385 (local address 127.15.75.190:36831)
0809 02:16:30.517520 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:30.517525 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:30.517541 (+ 16us) server_negotiation.cc:407] Connection header received
0809 02:16:30.517708 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:30.517711 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:30.517769 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:30.517872 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:30.518446 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:30.519046 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:30.519810 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:30.520024 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:30.520100 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:30.522608 (+ 2508us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.75.129:33385: BlockingRecv error: recv got EOF from 127.15.75.129:33385 (error 108)
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":80,"threads_started":1}
Aug 09 02:16:30 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.130@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:16:30 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.130@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:16:30.713049 29570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:30.707366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36831 (local address 127.15.75.130:36055)
0809 02:16:30.707631 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:30.707644 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:30.707742 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:30.707989 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:30.707991 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:30.708203 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:30.708313 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:30.708319 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:30.709112 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:30.709115 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:30.709673 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:30.709681 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:30.710088 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:30.710090 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:30.710153 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:30.710793 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:30.710809 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:30.712852 (+ 2043us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.75.190:36831: Server oryx/127.15.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":97,"threads_started":1}
W20260809 02:16:30.713088 29568 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:16:30.707474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.130:36055 (local address 127.15.75.190:36831)
0809 02:16:30.707613 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:30.707617 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:30.707710 (+ 93us) server_negotiation.cc:407] Connection header received
0809 02:16:30.707828 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:30.707830 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:30.707875 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:30.707951 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:30.708412 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:30.709009 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:30.709770 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:30.709984 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:30.710057 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:30.712985 (+ 2928us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.75.130:36055: BlockingRecv error: recv got EOF from 127.15.75.130:36055 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 09 02:16:30 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.131@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:16:30 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.131@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:16:30.921149 29573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:30.915286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36831 (local address 127.15.75.131:52249)
0809 02:16:30.915598 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:30.915611 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:30.915721 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:30.915960 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:30.915963 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:30.916148 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:30.916259 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:30.916264 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:30.917064 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:30.917067 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:30.917822 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:30.917835 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:30.918286 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:30.918293 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:30.918399 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:30.919097 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:30.919109 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:30.921007 (+ 1898us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.75.190:36831: Server oryx/127.15.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":96,"threads_started":1}
W20260809 02:16:30.921301 29568 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:16:30.915448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.131:52249 (local address 127.15.75.190:36831)
0809 02:16:30.915608 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:30.915611 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:30.915685 (+ 74us) server_negotiation.cc:407] Connection header received
0809 02:16:30.915803 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:30.915805 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:30.915853 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:30.915925 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:30.916353 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:30.916942 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:30.917946 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:30.918171 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:30.918251 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:30.921144 (+ 2893us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.75.131:52249: BlockingRecv error: recv got EOF from 127.15.75.131:52249 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 09 02:16:31 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.129@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:16:31 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.129@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:16:31.529379 29574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:31.523363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36831 (local address 127.15.75.129:36149)
0809 02:16:31.523637 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:31.523650 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:31.523749 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:31.524046 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:31.524049 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:31.524244 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:31.524363 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:31.524369 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:31.525117 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:31.525120 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:31.525779 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:31.525788 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:31.526435 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:31.526438 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:31.526555 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:31.527153 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:31.527175 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:31.529252 (+ 2077us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.75.190:36831: Server oryx/127.15.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":91,"threads_started":1}
W20260809 02:16:31.529539 29575 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:16:31.523448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.129:36149 (local address 127.15.75.190:36831)
0809 02:16:31.523683 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:31.523687 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:31.523702 (+ 15us) server_negotiation.cc:407] Connection header received
0809 02:16:31.523841 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:31.523843 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:31.523889 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:31.523970 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:31.524474 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:31.525022 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:31.525923 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:31.526285 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:31.526413 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:31.529400 (+ 2987us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.75.129:36149: BlockingRecv error: recv got EOF from 127.15.75.129:36149 (error 108)
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":59,"threads_started":1}
Aug 09 02:16:31 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.130@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:16:31 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.130@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:16:31.719127 29576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:31.713880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36831 (local address 127.15.75.130:50461)
0809 02:16:31.714179 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:31.714192 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:31.714294 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:31.714538 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:31.714540 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:31.714718 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:31.714818 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:31.714824 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:31.715538 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:31.715541 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:31.716091 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:31.716098 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:31.716506 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:31.716509 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:31.716578 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:31.717180 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:31.717195 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:31.719012 (+ 1817us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.75.190:36831: Server oryx/127.15.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":86,"threads_started":1}
W20260809 02:16:31.719300 29575 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:16:31.714044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.130:50461 (local address 127.15.75.190:36831)
0809 02:16:31.714202 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:31.714206 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:31.714243 (+ 37us) server_negotiation.cc:407] Connection header received
0809 02:16:31.714385 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:31.714387 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:31.714435 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:31.714503 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:31.714932 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:31.715446 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:31.716191 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:31.716401 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:31.716474 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:31.719143 (+ 2669us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.75.130:50461: BlockingRecv error: recv got EOF from 127.15.75.130:50461 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 09 02:16:31 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.131@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:16:31 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.131@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:16:31.927664 29577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:31.921918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36831 (local address 127.15.75.131:60567)
0809 02:16:31.922267 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:31.922289 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:31.922436 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:31.922748 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:31.922751 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:31.922959 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:31.923054 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:31.923059 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:31.923858 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:31.923863 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:31.924426 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:31.924434 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:31.924974 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:31.924978 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:31.925069 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:31.925634 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:31.925646 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:31.927492 (+ 1846us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.75.190:36831: Server oryx/127.15.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":92,"threads_started":1}
W20260809 02:16:31.927726 29575 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:16:31.922034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.131:60567 (local address 127.15.75.190:36831)
0809 02:16:31.922234 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:31.922239 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:31.922373 (+ 134us) server_negotiation.cc:407] Connection header received
0809 02:16:31.922549 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:31.922552 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:31.922610 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:31.922707 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:31.923177 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:31.923742 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:31.924547 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:31.924861 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:31.924972 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:31.927612 (+ 2640us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.75.131:60567: BlockingRecv error: recv got EOF from 127.15.75.131:60567 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
Aug 09 02:16:32 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.129@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:16:32 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.129@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:16:32.535903 29579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:32.530160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36831 (local address 127.15.75.129:55573)
0809 02:16:32.530472 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:32.530486 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:32.530579 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:32.530840 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:32.530842 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:32.531028 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:32.531132 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:32.531137 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:32.531868 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:32.531871 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:32.532438 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:32.532446 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:32.532921 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:32.532924 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:32.533031 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:32.533610 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:32.533622 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:32.535745 (+ 2123us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.75.190:36831: Server oryx/127.15.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":82,"threads_started":1}
W20260809 02:16:32.536115 29580 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:16:32.530219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.129:55573 (local address 127.15.75.190:36831)
0809 02:16:32.530526 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:32.530530 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:32.530543 (+ 13us) server_negotiation.cc:407] Connection header received
0809 02:16:32.530667 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:32.530670 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:32.530715 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:32.530790 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:32.531233 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:32.531765 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:32.532548 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:32.532799 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:32.532893 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:32.535957 (+ 3064us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.75.129:55573: BlockingRecv error: recv got EOF from 127.15.75.129:55573 (error 108)
Metrics: {"server-negotiator.queue_time_us":236,"thread_start_us":134,"threads_started":1}
Aug 09 02:16:32 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.130@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:16:32 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.130@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:16:32.725822 29581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:32.720030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36831 (local address 127.15.75.130:59703)
0809 02:16:32.720350 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:32.720363 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:32.720481 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:32.720735 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:32.720737 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:32.720924 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:32.721021 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:32.721026 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:32.721746 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:32.721748 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:32.722401 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:32.722410 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:32.723004 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:32.723008 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:32.723103 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:32.723698 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:32.723710 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:32.725683 (+ 1973us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.75.190:36831: Server oryx/127.15.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":79,"threads_started":1}
W20260809 02:16:32.725957 29580 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:16:32.720225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.130:59703 (local address 127.15.75.190:36831)
0809 02:16:32.720371 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:32.720375 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:32.720448 (+ 73us) server_negotiation.cc:407] Connection header received
0809 02:16:32.720561 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:32.720564 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:32.720608 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:32.720674 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:32.721118 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:32.721644 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:32.722511 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:32.722878 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:32.722990 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:32.725814 (+ 2824us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.75.130:59703: BlockingRecv error: recv got EOF from 127.15.75.130:59703 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 09 02:16:32 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.131@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:16:32 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.131@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:16:32.934463 29583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:32.928496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36831 (local address 127.15.75.131:33195)
0809 02:16:32.928850 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:32.928868 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:32.929012 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:32.929422 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:32.929425 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:32.929666 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:32.929772 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:32.929778 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:32.930568 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:32.930571 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:32.931128 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:32.931136 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:32.931533 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:32.931535 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:32.931595 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:32.932175 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:32.932187 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:32.934334 (+ 2147us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.75.190:36831: Server oryx/127.15.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":107,"threads_started":1}
W20260809 02:16:32.934726 29580 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:16:32.928600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.131:33195 (local address 127.15.75.190:36831)
0809 02:16:32.928757 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:32.928764 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:32.928954 (+ 190us) server_negotiation.cc:407] Connection header received
0809 02:16:32.929118 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:32.929122 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:32.929183 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:32.929259 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:32.929921 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:32.930473 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:32.931226 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:32.931438 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:32.931512 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:32.934489 (+ 2977us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.75.131:33195: BlockingRecv error: recv got EOF from 127.15.75.131:33195 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 09 02:16:33 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.129@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:16:33 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.129@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:16:33.542920 29584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:33.536899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36831 (local address 127.15.75.129:37325)
0809 02:16:33.537225 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:33.537243 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:33.537350 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:33.537673 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:33.537677 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:33.537939 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:33.538072 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:33.538087 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:33.538918 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:33.538920 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:33.539503 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:33.539511 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:33.539971 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:33.539973 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:33.540035 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:33.540528 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:33.540539 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:33.542802 (+ 2263us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.75.190:36831: Server oryx/127.15.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":98,"threads_started":1}
W20260809 02:16:33.543085 29585 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:16:33.536955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.129:37325 (local address 127.15.75.190:36831)
0809 02:16:33.537277 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:33.537283 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:33.537300 (+ 17us) server_negotiation.cc:407] Connection header received
0809 02:16:33.537471 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:33.537475 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:33.537534 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:33.537628 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:33.538193 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:33.538824 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:33.539614 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:33.539870 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:33.539952 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:33.542945 (+ 2993us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.75.129:37325: BlockingRecv error: recv got EOF from 127.15.75.129:37325 (error 108)
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":101,"threads_started":1}
Aug 09 02:16:33 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.130@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:16:33 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.130@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:16:33.732278 29588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:33.726586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36831 (local address 127.15.75.130:44519)
0809 02:16:33.726897 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:33.726935 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:33.727045 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:33.727346 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:33.727350 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:33.727573 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:33.727668 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:33.727674 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:33.728456 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:33.728460 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:33.729080 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:33.729090 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:33.729554 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:33.729557 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:33.729648 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:33.730140 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:33.730152 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:33.732106 (+ 1954us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.75.190:36831: Server oryx/127.15.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":75,"threads_started":1}
W20260809 02:16:33.732342 29585 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:16:33.726876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.130:44519 (local address 127.15.75.190:36831)
0809 02:16:33.727021 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:33.727025 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:33.727039 (+ 14us) server_negotiation.cc:407] Connection header received
0809 02:16:33.727164 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:33.727168 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:33.727222 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:33.727313 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:33.727786 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:33.728333 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:33.729204 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:33.729436 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:33.729548 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:33.732234 (+ 2686us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.75.130:44519: BlockingRecv error: recv got EOF from 127.15.75.130:44519 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 09 02:16:33 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.131@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:16:33 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.131@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:16:33.941159 29590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:33.935293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36831 (local address 127.15.75.131:47723)
0809 02:16:33.935588 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:33.935601 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:33.935699 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:33.935976 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:33.935978 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:33.936171 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:33.936272 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:33.936277 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:33.937044 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:33.937046 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:33.937712 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:33.937726 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:33.938217 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:33.938222 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:33.938303 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:33.938781 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:33.938792 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:33.940978 (+ 2186us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.75.190:36831: Server oryx/127.15.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":124,"threads_started":1}
W20260809 02:16:33.941252 29585 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:16:33.935402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.131:47723 (local address 127.15.75.190:36831)
0809 02:16:33.935540 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:33.935543 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:33.935671 (+ 128us) server_negotiation.cc:407] Connection header received
0809 02:16:33.935777 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:33.935780 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:33.935827 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:33.935902 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:33.936375 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:33.936940 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:33.937866 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:33.938089 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:33.938178 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:33.941117 (+ 2939us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.75.131:47723: BlockingRecv error: recv got EOF from 127.15.75.131:47723 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 09 02:16:34 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.129@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:16:34 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.129@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:16:34.554855 29591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:34.543752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36831 (local address 127.15.75.129:32943)
0809 02:16:34.544040 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:34.544053 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:34.544154 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:34.544440 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:34.544442 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:34.544626 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:34.544737 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:34.544742 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:34.545489 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:34.545492 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:34.546050 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:34.546057 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:34.546499 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:34.546502 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:34.546570 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:34.552424 (+ 5854us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:34.552437 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:34.554619 (+ 2182us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.75.190:36831: Server oryx/127.15.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":93,"threads_started":1}
W20260809 02:16:34.554916 29592 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:16:34.543798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.129:32943 (local address 127.15.75.190:36831)
0809 02:16:34.544057 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:34.544061 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:34.544104 (+ 43us) server_negotiation.cc:407] Connection header received
0809 02:16:34.544257 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:34.544260 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:34.544315 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:34.544414 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:34.544854 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:34.545394 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:34.546151 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:34.546375 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:34.546484 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:34.554757 (+ 8273us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.75.129:32943: BlockingRecv error: recv got EOF from 127.15.75.129:32943 (error 108)
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":71,"threads_started":1}
Aug 09 02:16:34 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.130@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:16:34 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.130@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:16:34.738737 29593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:34.733179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36831 (local address 127.15.75.130:35515)
0809 02:16:34.733457 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:34.733475 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:34.733578 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:34.733830 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:34.733832 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:34.734073 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:34.734192 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:34.734199 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:34.735017 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:34.735020 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:34.735575 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:34.735583 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:34.736042 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:34.736045 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:34.736110 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:34.736593 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:34.736604 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:34.738620 (+ 2016us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.75.190:36831: Server oryx/127.15.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":111,"threads_started":1}
W20260809 02:16:34.738883 29592 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:16:34.733245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.130:35515 (local address 127.15.75.190:36831)
0809 02:16:34.733412 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:34.733421 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:34.733547 (+ 126us) server_negotiation.cc:407] Connection header received
0809 02:16:34.733692 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:34.733695 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:34.733745 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:34.733821 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:34.734314 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:34.734895 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:34.735684 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:34.735940 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:34.736020 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:34.738759 (+ 2739us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.75.130:35515: BlockingRecv error: recv got EOF from 127.15.75.130:35515 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 09 02:16:34 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.131@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:16:34 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.131@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:16:34.947710 29594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:34.941983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36831 (local address 127.15.75.131:47231)
0809 02:16:34.942307 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:34.942319 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:34.942454 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:34.942765 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:34.942769 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:34.943000 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:34.943091 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:34.943097 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:34.943876 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:34.943881 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:34.944452 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:34.944460 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:34.944989 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:34.944993 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:34.945085 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:34.945668 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:34.945680 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:34.947596 (+ 1916us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.75.190:36831: Server oryx/127.15.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":89,"threads_started":1}
W20260809 02:16:34.947901 29592 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:16:34.942076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.131:47231 (local address 127.15.75.190:36831)
0809 02:16:34.942211 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:34.942216 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:34.942401 (+ 185us) server_negotiation.cc:407] Connection header received
0809 02:16:34.942564 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:34.942569 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:34.942625 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:34.942746 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:34.943218 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:34.943760 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:34.944580 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:34.944871 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:34.944983 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:34.947763 (+ 2780us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.75.131:47231: BlockingRecv error: recv got EOF from 127.15.75.131:47231 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 09 02:16:35 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.129@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:16:35 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.129@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:16:35.561810 29595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:35.555557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36831 (local address 127.15.75.129:36359)
0809 02:16:35.555948 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:35.555962 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:35.556073 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:35.556318 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:35.556320 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:35.556515 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:35.556614 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:35.556620 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:35.557466 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:35.557470 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:35.558248 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:35.558259 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:35.558848 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:35.558852 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:35.558930 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:35.559585 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:35.559598 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:35.561689 (+ 2091us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.75.190:36831: Server oryx/127.15.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":92,"threads_started":1}
W20260809 02:16:35.561962 29599 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:16:35.555657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.129:36359 (local address 127.15.75.190:36831)
0809 02:16:35.555932 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:35.555935 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:35.556041 (+ 106us) server_negotiation.cc:407] Connection header received
0809 02:16:35.556149 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:35.556152 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:35.556197 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:35.556277 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:35.556727 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:35.557330 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:35.558390 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:35.558673 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:35.558755 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:35.561820 (+ 3065us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.75.129:36359: BlockingRecv error: recv got EOF from 127.15.75.129:36359 (error 108)
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":70,"threads_started":1}
Aug 09 02:16:35 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.130@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:16:35 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.130@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:16:35.745364 29600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:35.739582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36831 (local address 127.15.75.130:52049)
0809 02:16:35.739894 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:35.739908 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:35.740007 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:35.740223 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:35.740225 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:35.740415 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:35.740536 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:35.740542 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:35.741515 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:35.741519 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:35.742172 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:35.742182 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:35.742662 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:35.742666 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:35.742770 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:35.743278 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:35.743290 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:35.745188 (+ 1898us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.75.190:36831: Server oryx/127.15.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":96,"threads_started":1}
W20260809 02:16:35.745433 29599 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:16:35.739663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.130:52049 (local address 127.15.75.190:36831)
0809 02:16:35.739813 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:35.739816 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:35.739956 (+ 140us) server_negotiation.cc:407] Connection header received
0809 02:16:35.740072 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:35.740075 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:35.740116 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:35.740185 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:35.740665 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:35.741395 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:35.742303 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:35.742535 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:35.742646 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:35.745322 (+ 2676us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.75.130:52049: BlockingRecv error: recv got EOF from 127.15.75.130:52049 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 09 02:16:35 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.131@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:16:35 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.131@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:16:35.954013 29602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:35.948631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36831 (local address 127.15.75.131:46329)
0809 02:16:35.949054 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:35.949067 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:35.949172 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:35.949405 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:35.949407 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:35.949595 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:35.949690 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:35.949695 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:35.950408 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:35.950410 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:35.950967 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:35.950974 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:35.951392 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:35.951394 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:35.951459 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:35.951949 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:35.951961 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:35.953843 (+ 1882us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.75.190:36831: Server oryx/127.15.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":132,"threads_started":1}
W20260809 02:16:35.954221 29599 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:16:35.948650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.131:46329 (local address 127.15.75.190:36831)
0809 02:16:35.948923 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:35.948928 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:35.949173 (+ 245us) server_negotiation.cc:407] Connection header received
0809 02:16:35.949242 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:35.949244 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:35.949295 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:35.949379 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:35.949789 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:35.950309 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:35.951068 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:35.951284 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:35.951360 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:35.954083 (+ 2723us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.75.131:46329: BlockingRecv error: recv got EOF from 127.15.75.131:46329 (error 108)
Metrics: {"server-negotiator.queue_time_us":175}
Aug 09 02:16:36 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.129@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:16:36 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.129@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:16:36.568908 29603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:36.562718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36831 (local address 127.15.75.129:57523)
0809 02:16:36.563056 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:36.563071 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:36.563162 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:36.563465 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:36.563468 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:36.563656 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:36.563773 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:36.563779 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:36.564612 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:36.564614 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:36.565222 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:36.565231 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:36.565643 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:36.565645 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:36.565707 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:36.566319 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:36.566333 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:36.568660 (+ 2327us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.75.190:36831: Server oryx/127.15.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":114,"threads_started":1}
W20260809 02:16:36.569253 29605 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:16:36.562826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.129:57523 (local address 127.15.75.190:36831)
0809 02:16:36.563103 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:36.563106 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:36.563120 (+ 14us) server_negotiation.cc:407] Connection header received
0809 02:16:36.563250 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:36.563252 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:36.563307 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:36.563388 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:36.563911 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:36.564498 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:36.565323 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:36.565545 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:36.565622 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:36.569119 (+ 3497us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.75.129:57523: BlockingRecv error: recv got EOF from 127.15.75.129:57523 (error 108)
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":120,"threads_started":1}
Aug 09 02:16:36 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.130@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:16:36 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.130@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:16:36.752362 29606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:36.746289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36831 (local address 127.15.75.130:56981)
0809 02:16:36.746594 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:36.746606 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:36.746743 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:36.747038 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:36.747041 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:36.747264 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:36.747361 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:36.747367 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:36.748301 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:36.748306 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:36.748973 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:36.748983 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:36.749572 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:36.749576 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:36.749683 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:36.750350 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:36.750366 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:36.752243 (+ 1877us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.75.190:36831: Server oryx/127.15.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":89,"threads_started":1}
W20260809 02:16:36.752535 29605 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:16:36.746348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.130:56981 (local address 127.15.75.190:36831)
0809 02:16:36.746524 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:36.746528 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:36.746686 (+ 158us) server_negotiation.cc:407] Connection header received
0809 02:16:36.746852 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:36.746855 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:36.746910 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:36.747007 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:36.747492 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:36.748181 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:36.749098 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:36.749428 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:36.749549 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:36.752377 (+ 2828us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.75.130:56981: BlockingRecv error: recv got EOF from 127.15.75.130:56981 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 09 02:16:36 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.131@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:16:36 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.131@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:16:36.960481 29607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:36.954741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36831 (local address 127.15.75.131:36873)
0809 02:16:36.955052 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:36.955065 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:36.955165 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:36.955409 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:36.955411 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:36.955629 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:36.955771 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:36.955778 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:36.956739 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:36.956742 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:36.957410 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:36.957420 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:36.957896 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:36.957900 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:36.957987 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:36.958510 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:36.958522 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:36.960308 (+ 1786us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.75.190:36831: Server oryx/127.15.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":136,"threads_started":1}
W20260809 02:16:36.960539 29605 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:16:36.954828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.131:36873 (local address 127.15.75.190:36831)
0809 02:16:36.954963 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:36.954966 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:36.955132 (+ 166us) server_negotiation.cc:407] Connection header received
0809 02:16:36.955247 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:36.955250 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:36.955297 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:36.955388 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:36.955895 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:36.956591 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:36.957538 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:36.957774 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:36.957892 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:36.960429 (+ 2537us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.75.131:36873: BlockingRecv error: recv got EOF from 127.15.75.131:36873 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 09 02:16:37 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.129@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:16:37 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.129@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:16:37.575429 29608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:37.569893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36831 (local address 127.15.75.129:50207)
0809 02:16:37.570194 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:37.570209 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:37.570316 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:37.570573 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:37.570575 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:37.570774 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:37.570888 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:37.570894 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:37.571651 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:37.571653 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:37.572244 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:37.572253 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:37.572749 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:37.572755 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:37.572827 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:37.573375 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:37.573386 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:37.575303 (+ 1917us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.75.190:36831: Server oryx/127.15.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":97,"threads_started":1}
W20260809 02:16:37.575652 29609 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:16:37.569960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.129:50207 (local address 127.15.75.190:36831)
0809 02:16:37.570230 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:37.570233 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:37.570272 (+ 39us) server_negotiation.cc:407] Connection header received
0809 02:16:37.570408 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:37.570410 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:37.570456 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:37.570534 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:37.570999 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:37.571557 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:37.572355 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:37.572595 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:37.572670 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:37.575454 (+ 2784us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.75.129:50207: BlockingRecv error: recv got EOF from 127.15.75.129:50207 (error 108)
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":66,"threads_started":1}
Aug 09 02:16:37 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.130@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:16:37 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.130@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:16:37.758679 29613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:37.753135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36831 (local address 127.15.75.130:37775)
0809 02:16:37.753445 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:37.753459 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:37.753559 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:37.753790 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:37.753792 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:37.753996 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:37.754116 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:37.754196 (+ 80us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:37.754964 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:37.754966 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:37.755521 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:37.755528 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:37.755960 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:37.755963 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:37.756023 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:37.756514 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:37.756526 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:37.758539 (+ 2013us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.75.190:36831: Server oryx/127.15.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":129,"threads_started":1}
W20260809 02:16:37.758823 29609 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:16:37.753170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.130:37775 (local address 127.15.75.190:36831)
0809 02:16:37.753336 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:37.753341 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:37.753534 (+ 193us) server_negotiation.cc:407] Connection header received
0809 02:16:37.753638 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:37.753641 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:37.753688 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:37.753755 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:37.754329 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:37.754861 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:37.755628 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:37.755857 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:37.755939 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:37.758673 (+ 2734us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.75.130:37775: BlockingRecv error: recv got EOF from 127.15.75.130:37775 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 09 02:16:37 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.131@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:16:37 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.131@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:16:37.966894 29614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:37.961363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36831 (local address 127.15.75.131:50287)
0809 02:16:37.961629 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:37.961642 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:37.961744 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:37.961981 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:37.961983 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:37.962184 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:37.962282 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:37.962287 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:37.963153 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:37.963156 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:37.963738 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:37.963747 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:37.964186 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:37.964189 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:37.964248 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:37.964762 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:37.964773 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:37.966719 (+ 1946us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.75.190:36831: Server oryx/127.15.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":95,"threads_started":1}
W20260809 02:16:37.967038 29609 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:16:37.961466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.131:50287 (local address 127.15.75.190:36831)
0809 02:16:37.961638 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:37.961642 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:37.961712 (+ 70us) server_negotiation.cc:407] Connection header received
0809 02:16:37.961822 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:37.961825 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:37.961872 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:37.961943 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:37.962379 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:37.963052 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:37.963862 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:37.964082 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:37.964170 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:37.966892 (+ 2722us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.75.131:50287: BlockingRecv error: recv got EOF from 127.15.75.131:50287 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Aug 09 02:16:38 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.129@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:16:38 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.129@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:16:38.581945 29615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:38.576371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36831 (local address 127.15.75.129:59799)
0809 02:16:38.576834 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:38.576847 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:38.576940 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:38.577226 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:38.577229 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:38.577444 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:38.577542 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:38.577548 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:38.578336 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:38.578340 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:38.578897 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:38.578905 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:38.579363 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:38.579367 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:38.579467 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:38.579927 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:38.579938 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:38.581774 (+ 1836us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.75.190:36831: Server oryx/127.15.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":102,"threads_started":1}
W20260809 02:16:38.582041 29616 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:16:38.576443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.129:59799 (local address 127.15.75.190:36831)
0809 02:16:38.576767 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:38.576772 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:38.576926 (+ 154us) server_negotiation.cc:407] Connection header received
0809 02:16:38.577047 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:38.577050 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:38.577110 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:38.577202 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:38.577663 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:38.578222 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:38.579016 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:38.579249 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:38.579359 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:38.581910 (+ 2551us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.75.129:59799: BlockingRecv error: recv got EOF from 127.15.75.129:59799 (error 108)
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":87,"threads_started":1}
Aug 09 02:16:38 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.130@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:16:38 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.130@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:16:38.765203 29618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:38.759450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36831 (local address 127.15.75.130:59221)
0809 02:16:38.759786 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:38.759801 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:38.759944 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:38.760244 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:38.760248 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:38.760486 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:38.760637 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:38.760645 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:38.761458 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:38.761461 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:38.762065 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:38.762078 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:38.762532 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:38.762535 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:38.762600 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:38.763150 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:38.763171 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:38.765088 (+ 1917us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.75.190:36831: Server oryx/127.15.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":132,"threads_started":1}
W20260809 02:16:38.765342 29616 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:16:38.759545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.130:59221 (local address 127.15.75.190:36831)
0809 02:16:38.759688 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:38.759693 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:38.759911 (+ 218us) server_negotiation.cc:407] Connection header received
0809 02:16:38.760039 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:38.760045 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:38.760106 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:38.760207 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:38.760786 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:38.761325 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:38.762178 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:38.762404 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:38.762528 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:38.765223 (+ 2695us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.75.130:59221: BlockingRecv error: recv got EOF from 127.15.75.130:59221 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 09 02:16:38 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.131@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:16:38 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.131@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:16:38.973080 29621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:38.967793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36831 (local address 127.15.75.131:33807)
0809 02:16:38.968092 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:38.968106 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:38.968198 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:38.968425 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:38.968427 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:38.968611 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:38.968735 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:38.968740 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:38.969438 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:38.969440 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:38.969981 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:38.969988 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:38.970382 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:38.970385 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:38.970452 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:38.970930 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:38.970941 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:38.972958 (+ 2017us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.75.190:36831: Server oryx/127.15.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":118,"threads_started":1}
W20260809 02:16:38.973222 29616 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:16:38.967925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.131:33807 (local address 127.15.75.190:36831)
0809 02:16:38.968065 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:38.968069 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:38.968174 (+ 105us) server_negotiation.cc:407] Connection header received
0809 02:16:38.968264 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:38.968266 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:38.968308 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:38.968374 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:38.968827 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:38.969341 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:38.970079 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:38.970287 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:38.970361 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:38.973105 (+ 2744us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.75.131:33807: BlockingRecv error: recv got EOF from 127.15.75.131:33807 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 09 02:16:39 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.129@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:16:39 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.129@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:16:39.591208 29622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:39.582774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36831 (local address 127.15.75.129:59525)
0809 02:16:39.583091 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:39.583106 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:39.583219 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:39.583473 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:39.583475 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:39.583671 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:39.583774 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:39.583779 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:39.584555 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:39.584558 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:39.585359 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:39.585369 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:39.585914 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:39.585917 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:39.585982 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:39.589210 (+ 3228us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:39.589230 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:39.591077 (+ 1847us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.75.190:36831: Server oryx/127.15.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":111,"threads_started":1}
W20260809 02:16:39.591346 29625 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:16:39.582841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.129:59525 (local address 127.15.75.190:36831)
0809 02:16:39.583082 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:39.583086 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:39.583188 (+ 102us) server_negotiation.cc:407] Connection header received
0809 02:16:39.583299 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:39.583301 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:39.583347 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:39.583435 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:39.583877 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:39.584416 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:39.585470 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:39.585805 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:39.585888 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:39.591212 (+ 5324us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.75.129:59525: BlockingRecv error: recv got EOF from 127.15.75.129:59525 (error 108)
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":89,"threads_started":1}
Aug 09 02:16:39 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.130@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:16:39 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.130@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:16:39.772240 29628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:39.766006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36831 (local address 127.15.75.130:33295)
0809 02:16:39.766340 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:39.766356 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:39.766513 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:39.766786 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:39.766789 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:39.767074 (+ 285us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:39.767208 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:39.767216 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:39.768086 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:39.768092 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:39.769066 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:39.769077 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:39.769694 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:39.769697 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:39.769760 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:39.770234 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:39.770245 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:39.772122 (+ 1877us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.75.190:36831: Server oryx/127.15.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":89,"threads_started":1}
W20260809 02:16:39.772521 29625 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:16:39.766137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.130:33295 (local address 127.15.75.190:36831)
0809 02:16:39.766317 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:39.766322 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:39.766446 (+ 124us) server_negotiation.cc:407] Connection header received
0809 02:16:39.766621 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:39.766624 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:39.766679 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:39.766764 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:39.767345 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:39.767976 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:39.769215 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:39.769602 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:39.769690 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:39.772272 (+ 2582us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.75.130:33295: BlockingRecv error: recv got EOF from 127.15.75.130:33295 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 09 02:16:39 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.131@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:16:39 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.131@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:16:39.979527 29629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:39.974007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36831 (local address 127.15.75.131:52417)
0809 02:16:39.974377 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:39.974391 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:39.974502 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:39.974750 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:39.974753 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:39.974944 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:39.975044 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:39.975049 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:39.975795 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:39.975797 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:39.976344 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:39.976351 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:39.976800 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:39.976803 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:39.976880 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:39.977458 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:39.977471 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:39.979411 (+ 1940us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.75.190:36831: Server oryx/127.15.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":111,"threads_started":1}
W20260809 02:16:39.979674 29625 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:16:39.974066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.131:52417 (local address 127.15.75.190:36831)
0809 02:16:39.974229 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:39.974234 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:39.974495 (+ 261us) server_negotiation.cc:407] Connection header received
0809 02:16:39.974573 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:39.974576 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:39.974627 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:39.974711 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:39.975163 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:39.975700 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:39.976453 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:39.976673 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:39.976772 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:39.979553 (+ 2781us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.75.131:52417: BlockingRecv error: recv got EOF from 127.15.75.131:52417 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 09 02:16:40 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.129@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:16:40 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.129@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:16:40.597502 29630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:40.592053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36831 (local address 127.15.75.129:34901)
0809 02:16:40.592372 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:40.592388 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:40.592491 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:40.592852 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:40.592855 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:40.593058 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:40.593167 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:40.593172 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:40.593925 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:40.593927 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:40.594490 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:40.594497 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:40.594908 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:40.594911 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:40.594973 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:40.595457 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:40.595469 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:40.597379 (+ 1910us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.75.190:36831: Server oryx/127.15.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":115,"threads_started":1}
W20260809 02:16:40.597724 29631 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:16:40.592150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.129:34901 (local address 127.15.75.190:36831)
0809 02:16:40.592401 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:40.592405 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:40.592448 (+ 43us) server_negotiation.cc:407] Connection header received
0809 02:16:40.592579 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:40.592582 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:40.592628 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:40.592723 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:40.593279 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:40.593824 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:40.594592 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:40.594811 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:40.594887 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:40.597552 (+ 2665us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.75.129:34901: BlockingRecv error: recv got EOF from 127.15.75.129:34901 (error 108)
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":91,"threads_started":1}
Aug 09 02:16:40 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.130@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:16:40 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.130@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:16:40.778774 29632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:40.773062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36831 (local address 127.15.75.130:40653)
0809 02:16:40.773345 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:40.773359 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:40.773467 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:40.773704 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:40.773706 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:40.773926 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:40.774028 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:40.774034 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:40.774903 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:40.774907 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:40.775597 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:40.775608 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:40.776071 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:40.776074 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:40.776164 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:40.776668 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:40.776679 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:40.778601 (+ 1922us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.75.190:36831: Server oryx/127.15.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":101,"threads_started":1}
W20260809 02:16:40.778836 29631 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:16:40.773160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.130:40653 (local address 127.15.75.190:36831)
0809 02:16:40.773301 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:40.773305 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:40.773429 (+ 124us) server_negotiation.cc:407] Connection header received
0809 02:16:40.773544 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:40.773547 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:40.773594 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:40.773668 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:40.774160 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:40.774781 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:40.775723 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:40.775955 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:40.776067 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:40.778730 (+ 2663us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.75.130:40653: BlockingRecv error: recv got EOF from 127.15.75.130:40653 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 09 02:16:40 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.131@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:16:40 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.131@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:16:40.986886 29639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:40.980394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36831 (local address 127.15.75.131:51477)
0809 02:16:40.980893 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:40.980911 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:40.981084 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:40.981391 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:40.981394 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:40.981631 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:40.981734 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:40.981740 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:40.982603 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:40.982609 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:40.983252 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:40.983260 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:40.983820 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:40.983824 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:40.983931 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:40.984548 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:40.984561 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:40.986694 (+ 2133us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.75.190:36831: Server oryx/127.15.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":105,"threads_started":1}
W20260809 02:16:40.986967 29631 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:16:40.980579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.131:51477 (local address 127.15.75.190:36831)
0809 02:16:40.980885 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:40.980890 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:40.981014 (+ 124us) server_negotiation.cc:407] Connection header received
0809 02:16:40.981199 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:40.981203 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:40.981263 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:40.981367 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:40.981878 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:40.982472 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:40.983403 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:40.983659 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:40.983785 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:40.986844 (+ 3059us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.75.131:51477: BlockingRecv error: recv got EOF from 127.15.75.131:51477 (error 108)
Metrics: {"server-negotiator.queue_time_us":166}
Aug 09 02:16:41 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.129@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:16:41 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.129@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:16:41.603853 29640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:41.598370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36831 (local address 127.15.75.129:42793)
0809 02:16:41.598657 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:41.598672 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:41.598763 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:41.599016 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:41.599018 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:41.599208 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:41.599306 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:41.599311 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:41.600041 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:41.600043 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:41.600656 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:41.600668 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:41.601304 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:41.601308 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:41.601398 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:41.601970 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:41.601982 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:41.603740 (+ 1758us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.75.190:36831: Server oryx/127.15.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":92,"threads_started":1}
W20260809 02:16:41.604012 29641 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:16:41.598441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.129:42793 (local address 127.15.75.190:36831)
0809 02:16:41.598720 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:41.598724 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:41.598736 (+ 12us) server_negotiation.cc:407] Connection header received
0809 02:16:41.598851 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:41.598853 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:41.598897 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:41.598974 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:41.599402 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:41.599948 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:41.600819 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:41.601140 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:41.601265 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:41.603875 (+ 2610us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.75.129:42793: BlockingRecv error: recv got EOF from 127.15.75.129:42793 (error 108)
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":95,"threads_started":1}
Aug 09 02:16:41 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.130@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:16:41 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.130@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:16:41.785393 29642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:41.779571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36831 (local address 127.15.75.130:35387)
0809 02:16:41.779952 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:41.779971 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:41.780115 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:41.780405 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:41.780408 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:41.780630 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:41.780761 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:41.780795 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:41.781577 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:41.781582 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:41.782162 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:41.782170 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:41.782646 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:41.782650 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:41.782752 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:41.783276 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:41.783288 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:41.785251 (+ 1963us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.75.190:36831: Server oryx/127.15.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":150,"threads_started":1}
W20260809 02:16:41.785560 29641 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:16:41.779743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.130:35387 (local address 127.15.75.190:36831)
0809 02:16:41.779909 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:41.779914 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:41.780045 (+ 131us) server_negotiation.cc:407] Connection header received
0809 02:16:41.780214 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:41.780217 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:41.780285 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:41.780376 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:41.780914 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:41.781461 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:41.782302 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:41.782529 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:41.782643 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:41.785403 (+ 2760us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.75.130:35387: BlockingRecv error: recv got EOF from 127.15.75.130:35387 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 09 02:16:41 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.131@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:16:41 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.131@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:16:41.993306 29644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:41.987767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36831 (local address 127.15.75.131:39203)
0809 02:16:41.988084 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:41.988099 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:41.988215 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:41.988439 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:41.988441 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:41.988636 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:41.988818 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:41.988824 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:41.989546 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:41.989548 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:41.990118 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:41.990126 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:41.990535 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:41.990537 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:41.990598 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:41.991102 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:41.991114 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:41.993100 (+ 1986us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.75.190:36831: Server oryx/127.15.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":117,"threads_started":1}
W20260809 02:16:41.993345 29641 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:16:41.987989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.131:39203 (local address 127.15.75.190:36831)
0809 02:16:41.988120 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:41.988124 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:41.988166 (+ 42us) server_negotiation.cc:407] Connection header received
0809 02:16:41.988282 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:41.988284 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:41.988329 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:41.988404 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:41.988923 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:41.989452 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:41.990215 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:41.990437 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:41.990513 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:41.993229 (+ 2716us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.75.131:39203: BlockingRecv error: recv got EOF from 127.15.75.131:39203 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 09 02:16:42 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.129@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:16:42 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.129@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:16:42.610769 29647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:42.604765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36831 (local address 127.15.75.129:59519)
0809 02:16:42.605075 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:42.605090 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:42.605188 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:42.605463 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:42.605465 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:42.605671 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:42.605818 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:42.605825 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:42.606569 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:42.606572 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:42.607228 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:42.607238 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:42.607662 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:42.607665 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:42.607772 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:42.608280 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:42.608292 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:42.610648 (+ 2356us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.75.190:36831: Server oryx/127.15.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":115,"threads_started":1}
W20260809 02:16:42.610965 29648 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:16:42.604858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.129:59519 (local address 127.15.75.190:36831)
0809 02:16:42.605166 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:42.605169 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:42.605182 (+ 13us) server_negotiation.cc:407] Connection header received
0809 02:16:42.605282 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:42.605285 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:42.605329 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:42.605410 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:42.605929 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:42.606467 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:42.607342 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:42.607561 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:42.607637 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:42.610810 (+ 3173us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.75.129:59519: BlockingRecv error: recv got EOF from 127.15.75.129:59519 (error 108)
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":91,"threads_started":1}
Aug 09 02:16:42 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.130@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:16:42 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.130@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:16:42.791705 29649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:42.786125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36831 (local address 127.15.75.130:51061)
0809 02:16:42.786425 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:42.786442 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:42.786554 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:42.786844 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:42.786847 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:42.787066 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:42.787162 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:42.787167 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:42.787957 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:42.787962 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:42.788541 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:42.788550 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:42.789056 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:42.789059 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:42.789147 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:42.789659 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:42.789671 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:42.791506 (+ 1835us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.75.190:36831: Server oryx/127.15.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":97,"threads_started":1}
W20260809 02:16:42.791818 29648 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:16:42.786348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.130:51061 (local address 127.15.75.190:36831)
0809 02:16:42.786513 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:42.786517 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:42.786531 (+ 14us) server_negotiation.cc:407] Connection header received
0809 02:16:42.786668 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:42.786673 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:42.786725 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:42.786811 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:42.787289 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:42.787827 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:42.788665 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:42.788939 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:42.789052 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:42.791702 (+ 2650us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.75.130:51061: BlockingRecv error: recv got EOF from 127.15.75.130:51061 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 09 02:16:42 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.131@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:16:42 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.131@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:16:42.999497 29650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:42.994137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36831 (local address 127.15.75.131:41787)
0809 02:16:42.994455 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:42.994468 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:42.994570 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:42.994799 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:42.994802 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:42.994994 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:42.995094 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:42.995099 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:42.995818 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:42.995820 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:42.996400 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:42.996409 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:42.996830 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:42.996833 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:42.996900 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:42.997459 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:42.997470 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:42.999326 (+ 1856us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.75.190:36831: Server oryx/127.15.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":89,"threads_started":1}
W20260809 02:16:42.999594 29648 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:16:42.994226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.131:41787 (local address 127.15.75.190:36831)
0809 02:16:42.994368 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:42.994371 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:42.994539 (+ 168us) server_negotiation.cc:407] Connection header received
0809 02:16:42.994638 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:42.994640 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:42.994686 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:42.994754 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:42.995194 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:42.995721 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:42.996505 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:42.996738 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:42.996813 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:42.999471 (+ 2658us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.75.131:41787: BlockingRecv error: recv got EOF from 127.15.75.131:41787 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 09 02:16:43 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.129@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:16:43 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.129@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:16:43.617050 29652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:43.611686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36831 (local address 127.15.75.129:53171)
0809 02:16:43.611973 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:43.611987 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:43.612073 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:43.612296 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:43.612299 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:43.612493 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:43.612595 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:43.612600 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:43.613400 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:43.613403 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:43.613979 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:43.613988 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:43.614418 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:43.614421 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:43.614483 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:43.615049 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:43.615061 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:43.616938 (+ 1877us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.75.190:36831: Server oryx/127.15.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":120,"threads_started":1}
W20260809 02:16:43.617235 29653 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:16:43.611759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.129:53171 (local address 127.15.75.190:36831)
0809 02:16:43.612036 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:43.612039 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:43.612053 (+ 14us) server_negotiation.cc:407] Connection header received
0809 02:16:43.612142 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:43.612144 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:43.612186 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:43.612262 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:43.612718 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:43.613283 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:43.614087 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:43.614309 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:43.614396 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:43.617064 (+ 2668us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.75.129:53171: BlockingRecv error: recv got EOF from 127.15.75.129:53171 (error 108)
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":83,"threads_started":1}
Aug 09 02:16:43 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.130@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:16:43 dist-test-slave-s11t krb5kdc[27987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.75.130@KRBTEST.COM for oryx/127.15.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:16:43.797861 29656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:43.792739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36831 (local address 127.15.75.130:57753)
0809 02:16:43.793009 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:43.793024 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:43.793127 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:43.793361 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:43.793364 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:43.793556 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:43.793653 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:43.793658 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:43.794375 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:43.794377 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:43.794967 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:43.794975 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:43.795378 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:43.795381 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:43.795442 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:43.795916 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:43.795928 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:43.797715 (+ 1787us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.75.190:36831: Server oryx/127.15.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":96,"threads_started":1}
W20260809 02:16:43.797942 29653 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:16:43.792807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.130:57753 (local address 127.15.75.190:36831)
0809 02:16:43.792953 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:43.792957 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:43.793102 (+ 145us) server_negotiation.cc:407] Connection header received
0809 02:16:43.793207 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:43.793209 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:43.793252 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:43.793321 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:43.793751 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:43.794274 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:43.795070 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:43.795282 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:43.795355 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:43.797849 (+ 2494us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.75.130:57753: BlockingRecv error: recv got EOF from 127.15.75.130:57753 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 02:16:43.902217 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 28904
I20260809 02:16:43.919328 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 29154
I20260809 02:16:43.928138 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 29350
I20260809 02:16:43.938458 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 28798
2026-08-09T02:16:43Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18203 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-09T02:16:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:16:43Z Disabled control of system clock
I20260809 02:16:43.976873 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241741788589-15662-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241741788589-15662-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241741788589-15662-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:46687
--webserver_interface=127.15.75.190
--webserver_port=0
--builtin_ntp_servers=127.15.75.148:36303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.75.190:46687
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 02:16:44.088925 29665 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:16:44.089169 29665 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:16:44.089201 29665 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:16:44.093269 29665 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:16:44.093317 29665 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:16:44.093333 29665 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:16:44.093375 29665 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:16:44.093408 29665 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:16:44.098029 29665 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:36303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241741788589-15662-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241741788589-15662-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.75.190:46687
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:46687
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.75.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29665
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241741788589-15662-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:16:44.099135 29665 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:16:44.100476 29665 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:16:44.108978 29674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:44.109042 29677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:44.109062 29673 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:44.109157 29665 server_base.cc:1034] running on GCE node
I20260809 02:16:44.109810 29665 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:16:44.110855 29665 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:16:44.112049 29665 hybrid_clock.cc:648] HybridClock initialized: now 1786241804112032 us; error 32 us; skew 500 ppm
I20260809 02:16:44.113900 29665 webserver.cc:460] Webserver started at http://127.15.75.190:39155/ using document root <none> and password file <none>
I20260809 02:16:44.114666 29665 fs_manager.cc:359] Metadata directory not provided
I20260809 02:16:44.114753 29665 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:16:44.114997 29665 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:16:44.117095 29665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241741788589-15662-0/minicluster-data/master-0/data/instance:
uuid: "7a830ba07ff447f9912cdd203fdd8c08"
format_stamp: "Formatted at 2026-08-09 02:16:44 on dist-test-slave-s11t"
server_key: "dbe7947a51326ee98eb64ab3efa9414d"
server_key_iv: "dab3f28ec59c5d97a1ea665b67ffcfbe"
server_key_version: "encryptionkey@0"
I20260809 02:16:44.117663 29665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241741788589-15662-0/minicluster-data/master-0/wal/instance:
uuid: "7a830ba07ff447f9912cdd203fdd8c08"
format_stamp: "Formatted at 2026-08-09 02:16:44 on dist-test-slave-s11t"
server_key: "dbe7947a51326ee98eb64ab3efa9414d"
server_key_iv: "dab3f28ec59c5d97a1ea665b67ffcfbe"
server_key_version: "encryptionkey@0"
I20260809 02:16:44.121203 29665 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 02:16:44.123606 29688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:44.124812 29665 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 02:16:44.124945 29665 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241741788589-15662-0/minicluster-data/master-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241741788589-15662-0/minicluster-data/master-0/wal
uuid: "7a830ba07ff447f9912cdd203fdd8c08"
format_stamp: "Formatted at 2026-08-09 02:16:44 on dist-test-slave-s11t"
server_key: "dbe7947a51326ee98eb64ab3efa9414d"
server_key_iv: "dab3f28ec59c5d97a1ea665b67ffcfbe"
server_key_version: "encryptionkey@0"
I20260809 02:16:44.125056 29665 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241741788589-15662-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241741788589-15662-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241741788589-15662-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:16:44.144104 29665 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:16:44.144872 29665 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:16:44.145071 29665 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:16:44.152155 29665 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.190:46687
I20260809 02:16:44.152179 29779 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.190:46687 every 8 connection(s)
I20260809 02:16:44.153190 29665 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
I20260809 02:16:44.156265 29780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:16:44.161710 29780 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a830ba07ff447f9912cdd203fdd8c08: Bootstrap starting.
I20260809 02:16:44.162794 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 29665
I20260809 02:16:44.162891 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241741788589-15662-0/minicluster-data/master-0/wal/instance
I20260809 02:16:44.163203 15662 external_mini_cluster.cc:1428] Setting key f1cdbe507b1844c3a49c6099c5836b67
I20260809 02:16:44.164453 29780 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7a830ba07ff447f9912cdd203fdd8c08: Neither blocks nor log segments found. Creating new log.
I20260809 02:16:44.165241 29780 log.cc:824] T 00000000000000000000000000000000 P 7a830ba07ff447f9912cdd203fdd8c08: Log is configured to *not* fsync() on all Append() calls
I20260809 02:16:44.167291 29780 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a830ba07ff447f9912cdd203fdd8c08: No bootstrap required, opened a new log
I20260809 02:16:44.171553 29780 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7a830ba07ff447f9912cdd203fdd8c08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a830ba07ff447f9912cdd203fdd8c08" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 46687 } }
I20260809 02:16:44.171756 29780 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7a830ba07ff447f9912cdd203fdd8c08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:16:44.171810 29780 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7a830ba07ff447f9912cdd203fdd8c08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a830ba07ff447f9912cdd203fdd8c08, State: Initialized, Role: FOLLOWER
I20260809 02:16:44.172219 29780 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7a830ba07ff447f9912cdd203fdd8c08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a830ba07ff447f9912cdd203fdd8c08" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 46687 } }
I20260809 02:16:44.172334 29780 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7a830ba07ff447f9912cdd203fdd8c08 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:16:44.172422 29780 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7a830ba07ff447f9912cdd203fdd8c08 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:16:44.172537 29780 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7a830ba07ff447f9912cdd203fdd8c08 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:16:44.172650 29784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:44.164408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54486 (local address 127.15.75.190:46687)
0809 02:16:44.165098 (+ 690us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:44.165113 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:44.165147 (+ 34us) server_negotiation.cc:407] Connection header received
0809 02:16:44.165814 (+ 667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:44.165832 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:44.166155 (+ 323us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:44.166427 (+ 272us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:44.166748 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:44.167805 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:44.169034 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:44.169499 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:44.169620 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:44.170499 (+ 879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:44.170504 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:44.170516 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:16:44.170903 (+ 387us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:44.170959 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:16:44.170979 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:44.171056 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:44.171217 (+ 161us) server_negotiation.cc:299] Negotiation successful
0809 02:16:44.171501 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":157,"threads_started":1}
I20260809 02:16:44.173447 29780 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7a830ba07ff447f9912cdd203fdd8c08 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a830ba07ff447f9912cdd203fdd8c08" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 46687 } }
I20260809 02:16:44.173784 29780 leader_election.cc:302] T 00000000000000000000000000000000 P 7a830ba07ff447f9912cdd203fdd8c08 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7a830ba07ff447f9912cdd203fdd8c08; no voters:
I20260809 02:16:44.174137 29780 leader_election.cc:288] T 00000000000000000000000000000000 P 7a830ba07ff447f9912cdd203fdd8c08 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:16:44.174178 29786 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7a830ba07ff447f9912cdd203fdd8c08 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:16:44.174366 29786 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7a830ba07ff447f9912cdd203fdd8c08 [term 1 LEADER]: Becoming Leader. State: Replica: 7a830ba07ff447f9912cdd203fdd8c08, State: Running, Role: LEADER
I20260809 02:16:44.174662 29786 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7a830ba07ff447f9912cdd203fdd8c08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a830ba07ff447f9912cdd203fdd8c08" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 46687 } }
I20260809 02:16:44.175335 29780 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7a830ba07ff447f9912cdd203fdd8c08 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:16:44.175812 29789 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a830ba07ff447f9912cdd203fdd8c08 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7a830ba07ff447f9912cdd203fdd8c08" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a830ba07ff447f9912cdd203fdd8c08" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 46687 } } }
I20260809 02:16:44.175942 29789 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a830ba07ff447f9912cdd203fdd8c08 [sys.catalog]: This master's current role is: LEADER
I20260809 02:16:44.176290 29787 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a830ba07ff447f9912cdd203fdd8c08 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7a830ba07ff447f9912cdd203fdd8c08. Latest consensus state: current_term: 1 leader_uuid: "7a830ba07ff447f9912cdd203fdd8c08" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a830ba07ff447f9912cdd203fdd8c08" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 46687 } } }
I20260809 02:16:44.176362 29793 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:16:44.176424 29787 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a830ba07ff447f9912cdd203fdd8c08 [sys.catalog]: This master's current role is: LEADER
I20260809 02:16:44.179498 29793 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:16:44.185111 29793 catalog_manager.cc:1347] Generated new cluster ID: 26df7263d36c483782746203481b67d1
I20260809 02:16:44.185184 29793 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:16:44.206494 29793 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:16:44.207370 29793 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:16:44.213092 29793 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7a830ba07ff447f9912cdd203fdd8c08: Generated new TSK 0
I20260809 02:16:44.213730 29793 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:16:44.224275 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.129:0
--local_ip_for_outbound_sockets=127.15.75.129
--webserver_interface=127.15.75.129
--webserver_port=0
--tserver_master_addrs=127.15.75.190:46687
--builtin_ntp_servers=127.15.75.148:36303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260809 02:16:44.341427 29814 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:16:44.341645 29814 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:16:44.341681 29814 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:16:44.345635 29814 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:16:44.345679 29814 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:16:44.345784 29814 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.129
I20260809 02:16:44.350317 29814 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:36303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.75.129
--webserver_port=0
--tserver_master_addrs=127.15.75.190:46687
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29814
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.129
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:16:44.351296 29814 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:16:44.352592 29814 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:16:44.362079 29820 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:44.362097 29822 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:44.362195 29814 server_base.cc:1034] running on GCE node
W20260809 02:16:44.362097 29824 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:44.362953 29814 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:16:44.363590 29814 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:16:44.364760 29814 hybrid_clock.cc:648] HybridClock initialized: now 1786241804364748 us; error 29 us; skew 500 ppm
I20260809 02:16:44.366528 29814 webserver.cc:460] Webserver started at http://127.15.75.129:35371/ using document root <none> and password file <none>
I20260809 02:16:44.367156 29814 fs_manager.cc:359] Metadata directory not provided
I20260809 02:16:44.367249 29814 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:16:44.367470 29814 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:16:44.369553 29814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-0/data/instance:
uuid: "d6baa40a322f4b82a3a1f90cc1971c2b"
format_stamp: "Formatted at 2026-08-09 02:16:44 on dist-test-slave-s11t"
server_key: "736871fa3ea629c1758b837fa73ba069"
server_key_iv: "33ab178a1426348abc61fa262a8faa09"
server_key_version: "encryptionkey@0"
I20260809 02:16:44.370112 29814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-0/wal/instance:
uuid: "d6baa40a322f4b82a3a1f90cc1971c2b"
format_stamp: "Formatted at 2026-08-09 02:16:44 on dist-test-slave-s11t"
server_key: "736871fa3ea629c1758b837fa73ba069"
server_key_iv: "33ab178a1426348abc61fa262a8faa09"
server_key_version: "encryptionkey@0"
I20260809 02:16:44.373740 29814 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.001s
I20260809 02:16:44.376183 29830 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:44.377522 29814 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 02:16:44.377655 29814 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-0/wal
uuid: "d6baa40a322f4b82a3a1f90cc1971c2b"
format_stamp: "Formatted at 2026-08-09 02:16:44 on dist-test-slave-s11t"
server_key: "736871fa3ea629c1758b837fa73ba069"
server_key_iv: "33ab178a1426348abc61fa262a8faa09"
server_key_version: "encryptionkey@0"
I20260809 02:16:44.377767 29814 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:16:44.422096 29814 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:16:44.422873 29814 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:16:44.423082 29814 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:16:44.423715 29814 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:16:44.424743 29814 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:16:44.424814 29814 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:44.424896 29814 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:16:44.424942 29814 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:44.434284 29814 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.129:36845
I20260809 02:16:44.434322 29989 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.129:36845 every 8 connection(s)
I20260809 02:16:44.435194 29814 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
I20260809 02:16:44.442060 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 29814
I20260809 02:16:44.442153 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-0/wal/instance
I20260809 02:16:44.442512 15662 external_mini_cluster.cc:1428] Setting key 59425bd0148c03eb5fa1a9558d118a43
I20260809 02:16:44.444599 29784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:44.437051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.129:34751 (local address 127.15.75.190:46687)
0809 02:16:44.437170 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:44.437174 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:44.438259 (+ 1085us) server_negotiation.cc:407] Connection header received
0809 02:16:44.439637 (+ 1378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:44.439639 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:44.439685 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:44.439757 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:44.440563 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:44.441174 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:44.442390 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:44.442729 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:44.442837 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:44.444096 (+ 1259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:44.444100 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:44.444103 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:16:44.444118 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:44.444141 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:16:44.444151 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:44.444234 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:44.444369 (+ 135us) server_negotiation.cc:299] Negotiation successful
0809 02:16:44.444427 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 02:16:44.444808 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.130:0
--local_ip_for_outbound_sockets=127.15.75.130
--webserver_interface=127.15.75.130
--webserver_port=0
--tserver_master_addrs=127.15.75.190:46687
--builtin_ntp_servers=127.15.75.148:36303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260809 02:16:44.445502 29993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:44.437471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:46687 (local address 127.15.75.129:34751)
0809 02:16:44.438133 (+ 662us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:44.438173 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:44.439449 (+ 1276us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:44.439885 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:44.439891 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:44.439969 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:44.440446 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:44.440458 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:44.441291 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:44.441293 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:44.442270 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:44.442280 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:44.442891 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:44.442895 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:44.443133 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:44.443762 (+ 629us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:16:44.443764 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:44.443816 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:16:44.443935 (+ 119us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:16:44.444234 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:16:44.444240 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:16:44.444244 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:16:44.444336 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 02:16:44.444570 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":128,"threads_started":1}
I20260809 02:16:44.446699 29991 heartbeater.cc:347] Connected to a master server at 127.15.75.190:46687
I20260809 02:16:44.446928 29991 heartbeater.cc:464] Registering TS with master...
I20260809 02:16:44.447434 29991 heartbeater.cc:511] Master 127.15.75.190:46687 requested a full tablet report, sending...
I20260809 02:16:44.449044 29716 ts_manager.cc:194] Registered new tserver with Master: d6baa40a322f4b82a3a1f90cc1971c2b (127.15.75.129:36845)
I20260809 02:16:44.450775 29716 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.75.129:34751
W20260809 02:16:44.557430 29994 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:16:44.557682 29994 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:16:44.557718 29994 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:16:44.561899 29994 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:16:44.561957 29994 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:16:44.562009 29994 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.130
I20260809 02:16:44.566636 29994 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:36303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.75.130
--webserver_port=0
--tserver_master_addrs=127.15.75.190:46687
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29994
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.130
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:16:44.567720 29994 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:16:44.569139 29994 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:16:44.579226 30000 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:44.579270 30002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:44.579228 30004 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:44.579470 29994 server_base.cc:1034] running on GCE node
I20260809 02:16:44.580006 29994 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:16:44.580668 29994 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:16:44.581893 29994 hybrid_clock.cc:648] HybridClock initialized: now 1786241804581874 us; error 53 us; skew 500 ppm
I20260809 02:16:44.583813 29994 webserver.cc:460] Webserver started at http://127.15.75.130:40875/ using document root <none> and password file <none>
I20260809 02:16:44.584406 29994 fs_manager.cc:359] Metadata directory not provided
I20260809 02:16:44.584470 29994 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:16:44.584812 29994 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:16:44.586911 29994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-1/data/instance:
uuid: "c0e17864add540c8bc1b8e856a93963f"
format_stamp: "Formatted at 2026-08-09 02:16:44 on dist-test-slave-s11t"
server_key: "2796f093b4b16b75149db2804e0b60d5"
server_key_iv: "0e307121c2ce255392d6e7f6af1d4a13"
server_key_version: "encryptionkey@0"
I20260809 02:16:44.587477 29994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-1/wal/instance:
uuid: "c0e17864add540c8bc1b8e856a93963f"
format_stamp: "Formatted at 2026-08-09 02:16:44 on dist-test-slave-s11t"
server_key: "2796f093b4b16b75149db2804e0b60d5"
server_key_iv: "0e307121c2ce255392d6e7f6af1d4a13"
server_key_version: "encryptionkey@0"
I20260809 02:16:44.591266 29994 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260809 02:16:44.593883 30013 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:44.595053 29994 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 02:16:44.595189 29994 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-1/wal
uuid: "c0e17864add540c8bc1b8e856a93963f"
format_stamp: "Formatted at 2026-08-09 02:16:44 on dist-test-slave-s11t"
server_key: "2796f093b4b16b75149db2804e0b60d5"
server_key_iv: "0e307121c2ce255392d6e7f6af1d4a13"
server_key_version: "encryptionkey@0"
I20260809 02:16:44.595301 29994 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:16:44.612453 29994 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:16:44.613181 29994 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:16:44.613384 29994 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:16:44.613987 29994 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:16:44.614939 29994 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:16:44.615010 29994 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:44.615082 29994 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:16:44.615125 29994 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:44.624042 29994 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.130:46721
I20260809 02:16:44.624112 30192 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.130:46721 every 8 connection(s)
I20260809 02:16:44.625061 29994 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
I20260809 02:16:44.632597 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 29994
I20260809 02:16:44.632733 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-1/wal/instance
I20260809 02:16:44.633104 15662 external_mini_cluster.cc:1428] Setting key 0dbcdab99e9b415f3eb798aa64214aff
I20260809 02:16:44.634740 29784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:44.626888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.130:57115 (local address 127.15.75.190:46687)
0809 02:16:44.627049 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:44.627053 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:44.628096 (+ 1043us) server_negotiation.cc:407] Connection header received
0809 02:16:44.629653 (+ 1557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:44.629657 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:44.629725 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:44.629830 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:44.630718 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:44.631277 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:44.632483 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:44.632891 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:44.632976 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:44.634263 (+ 1287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:44.634267 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:44.634270 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:16:44.634283 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:44.634311 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:16:44.634321 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:44.634409 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:44.634583 (+ 174us) server_negotiation.cc:299] Negotiation successful
0809 02:16:44.634633 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 02:16:44.635306 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.131:0
--local_ip_for_outbound_sockets=127.15.75.131
--webserver_interface=127.15.75.131
--webserver_port=0
--tserver_master_addrs=127.15.75.190:46687
--builtin_ntp_servers=127.15.75.148:36303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260809 02:16:44.635910 30196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:44.627230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:46687 (local address 127.15.75.130:57115)
0809 02:16:44.627912 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:44.627960 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:44.629432 (+ 1472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:44.630006 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:44.630013 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:44.630093 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:44.630574 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:44.630586 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:44.631419 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:44.631423 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:44.632335 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:44.632347 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:44.633056 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:44.633063 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:44.633280 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:44.633992 (+ 712us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:16:44.633994 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:44.634035 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:16:44.634098 (+ 63us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:16:44.634414 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:16:44.634426 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:16:44.634435 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 02:16:44.634581 (+ 146us) client_negotiation.cc:240] Negotiation successful
0809 02:16:44.634860 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":129,"threads_started":1}
I20260809 02:16:44.637142 30193 heartbeater.cc:347] Connected to a master server at 127.15.75.190:46687
I20260809 02:16:44.637418 30193 heartbeater.cc:464] Registering TS with master...
I20260809 02:16:44.638058 30193 heartbeater.cc:511] Master 127.15.75.190:46687 requested a full tablet report, sending...
I20260809 02:16:44.639186 29716 ts_manager.cc:194] Registered new tserver with Master: c0e17864add540c8bc1b8e856a93963f (127.15.75.130:46721)
I20260809 02:16:44.639802 29716 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.75.130:57115
W20260809 02:16:44.750391 30198 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:16:44.750638 30198 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:16:44.750675 30198 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:16:44.754734 30198 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:16:44.754791 30198 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:16:44.754880 30198 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.131
I20260809 02:16:44.759697 30198 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:36303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.75.131
--webserver_port=0
--tserver_master_addrs=127.15.75.190:46687
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30198
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.131
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:16:44.760785 30198 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:16:44.762166 30198 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:16:44.772081 30206 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:44.772094 30205 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:44.772102 30210 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:44.772306 30198 server_base.cc:1034] running on GCE node
I20260809 02:16:44.772855 30198 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:16:44.773471 30198 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:16:44.774663 30198 hybrid_clock.cc:648] HybridClock initialized: now 1786241804774651 us; error 29 us; skew 500 ppm
I20260809 02:16:44.776561 30198 webserver.cc:460] Webserver started at http://127.15.75.131:40809/ using document root <none> and password file <none>
I20260809 02:16:44.777184 30198 fs_manager.cc:359] Metadata directory not provided
I20260809 02:16:44.777297 30198 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:16:44.777534 30198 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:16:44.779572 30198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-2/data/instance:
uuid: "ba9f4f24576f4e9a8d94bae633d82ded"
format_stamp: "Formatted at 2026-08-09 02:16:44 on dist-test-slave-s11t"
server_key: "ad283b1da85e78f44757b2bdeea590a3"
server_key_iv: "dd094b88ddc7cc8a88cb079362835137"
server_key_version: "encryptionkey@0"
I20260809 02:16:44.780135 30198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-2/wal/instance:
uuid: "ba9f4f24576f4e9a8d94bae633d82ded"
format_stamp: "Formatted at 2026-08-09 02:16:44 on dist-test-slave-s11t"
server_key: "ad283b1da85e78f44757b2bdeea590a3"
server_key_iv: "dd094b88ddc7cc8a88cb079362835137"
server_key_version: "encryptionkey@0"
I20260809 02:16:44.783854 30198 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260809 02:16:44.786448 30216 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:44.787583 30198 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 02:16:44.787720 30198 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-2/wal
uuid: "ba9f4f24576f4e9a8d94bae633d82ded"
format_stamp: "Formatted at 2026-08-09 02:16:44 on dist-test-slave-s11t"
server_key: "ad283b1da85e78f44757b2bdeea590a3"
server_key_iv: "dd094b88ddc7cc8a88cb079362835137"
server_key_version: "encryptionkey@0"
I20260809 02:16:44.787834 30198 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:16:44.807996 30198 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:16:44.808743 30198 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:16:44.808947 30198 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:16:44.809547 30198 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:16:44.810510 30198 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:16:44.810586 30198 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:44.810654 30198 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:16:44.810706 30198 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:44.819746 30198 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.131:33299
I20260809 02:16:44.819774 30398 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.131:33299 every 8 connection(s)
I20260809 02:16:44.820623 30198 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
I20260809 02:16:44.823448 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 30198
I20260809 02:16:44.823554 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-2/wal/instance
I20260809 02:16:44.823870 15662 external_mini_cluster.cc:1428] Setting key 87021137827452de6d7d9897c48fba89
I20260809 02:16:44.830184 29784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:44.822400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.131:58645 (local address 127.15.75.190:46687)
0809 02:16:44.822548 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:44.822552 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:44.823593 (+ 1041us) server_negotiation.cc:407] Connection header received
0809 02:16:44.825315 (+ 1722us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:44.825318 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:44.825363 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:44.825456 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:44.826432 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:44.826963 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:44.828204 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:44.828439 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:44.828545 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:44.829675 (+ 1130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:44.829679 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:44.829681 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:16:44.829702 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:44.829720 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:16:44.829727 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:44.829830 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:44.829987 (+ 157us) server_negotiation.cc:299] Negotiation successful
0809 02:16:44.830047 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 02:16:44.830978 30404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:44.822787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:46687 (local address 127.15.75.131:58645)
0809 02:16:44.823436 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:44.823475 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:44.825080 (+ 1605us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:44.825637 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:44.825644 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:44.825722 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:44.826297 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:44.826310 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:44.827090 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:44.827093 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:44.828080 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:44.828090 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:44.828594 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:44.828598 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:44.828833 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:44.829427 (+ 594us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:16:44.829428 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:44.829466 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:16:44.829527 (+ 61us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:16:44.829831 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:16:44.829844 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:16:44.829852 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 02:16:44.829986 (+ 134us) client_negotiation.cc:240] Negotiation successful
0809 02:16:44.830252 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":120,"threads_started":1}
I20260809 02:16:44.831979 30399 heartbeater.cc:347] Connected to a master server at 127.15.75.190:46687
I20260809 02:16:44.832207 30399 heartbeater.cc:464] Registering TS with master...
I20260809 02:16:44.832757 30399 heartbeater.cc:511] Master 127.15.75.190:46687 requested a full tablet report, sending...
I20260809 02:16:44.833736 29716 ts_manager.cc:194] Registered new tserver with Master: ba9f4f24576f4e9a8d94bae633d82ded (127.15.75.131:33299)
I20260809 02:16:44.834301 29716 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.75.131:58645
I20260809 02:16:44.837636 15662 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 02:16:44.840456 29784 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:16:44.839476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49536 (local address 127.15.75.190:46687)
0809 02:16:44.839569 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:44.839572 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:44.839813 (+ 241us) server_negotiation.cc:407] Connection header received
0809 02:16:44.839944 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:44.839947 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:44.840017 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:44.840109 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:44.840366 (+ 257us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49536: BlockingRecv error: recv got EOF from 127.0.0.1:49536 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 02:16:44.840808 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 29814
I20260809 02:16:44.860527 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 29994
I20260809 02:16:44.869800 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 30198
I20260809 02:16:44.881965 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 29665
2026-08-09T02:16:44Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (946 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-09T02:16:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:16:44Z Disabled control of system clock
I20260809 02:16:44.922513 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241741788589-15662-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241741788589-15662-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241741788589-15662-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:42127
--webserver_interface=127.15.75.190
--webserver_port=0
--builtin_ntp_servers=127.15.75.148:33831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.75.190:42127
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 02:16:45.033233 30419 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:16:45.033483 30419 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:16:45.033519 30419 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:16:45.037704 30419 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:16:45.037765 30419 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:16:45.037781 30419 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:16:45.037797 30419 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:16:45.037813 30419 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:16:45.042645 30419 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:33831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241741788589-15662-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241741788589-15662-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.75.190:42127
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:42127
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.75.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30419
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241741788589-15662-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:16:45.043730 30419 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:16:45.045151 30419 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:16:45.053669 30438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:45.053681 30432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:45.053687 30429 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:45.054104 30419 server_base.cc:1034] running on GCE node
I20260809 02:16:45.054613 30419 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:16:45.055658 30419 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:16:45.056846 30419 hybrid_clock.cc:648] HybridClock initialized: now 1786241805056819 us; error 45 us; skew 500 ppm
I20260809 02:16:45.058706 30419 webserver.cc:460] Webserver started at http://127.15.75.190:45691/ using document root <none> and password file <none>
I20260809 02:16:45.059276 30419 fs_manager.cc:359] Metadata directory not provided
I20260809 02:16:45.059370 30419 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:16:45.059604 30419 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:16:45.061645 30419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241741788589-15662-0/minicluster-data/master-0/data/instance:
uuid: "10844fceb3404bf68c58829948d7431d"
format_stamp: "Formatted at 2026-08-09 02:16:45 on dist-test-slave-s11t"
server_key: "8bb853b471941996269f2825f3d85212"
server_key_iv: "ea0035fa6d4290143618fee07f691305"
server_key_version: "encryptionkey@0"
I20260809 02:16:45.062208 30419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241741788589-15662-0/minicluster-data/master-0/wal/instance:
uuid: "10844fceb3404bf68c58829948d7431d"
format_stamp: "Formatted at 2026-08-09 02:16:45 on dist-test-slave-s11t"
server_key: "8bb853b471941996269f2825f3d85212"
server_key_iv: "ea0035fa6d4290143618fee07f691305"
server_key_version: "encryptionkey@0"
I20260809 02:16:45.065793 30419 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 02:16:45.068066 30447 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:45.069301 30419 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 02:16:45.069451 30419 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241741788589-15662-0/minicluster-data/master-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241741788589-15662-0/minicluster-data/master-0/wal
uuid: "10844fceb3404bf68c58829948d7431d"
format_stamp: "Formatted at 2026-08-09 02:16:45 on dist-test-slave-s11t"
server_key: "8bb853b471941996269f2825f3d85212"
server_key_iv: "ea0035fa6d4290143618fee07f691305"
server_key_version: "encryptionkey@0"
I20260809 02:16:45.069564 30419 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241741788589-15662-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241741788589-15662-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241741788589-15662-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:16:45.074056 30419 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:16:45.074652 30419 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:16:45.074846 30419 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:16:45.081799 30419 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.190:42127
I20260809 02:16:45.081810 30526 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.190:42127 every 8 connection(s)
I20260809 02:16:45.082692 30419 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
I20260809 02:16:45.086015 30527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:16:45.088207 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 30419
I20260809 02:16:45.088322 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241741788589-15662-0/minicluster-data/master-0/wal/instance
I20260809 02:16:45.088651 15662 external_mini_cluster.cc:1428] Setting key a192799e5bbe33bc0cb5020fd9f27838
I20260809 02:16:45.092097 30527 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10844fceb3404bf68c58829948d7431d: Bootstrap starting.
I20260809 02:16:45.094130 30533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:45.090076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44196 (local address 127.15.75.190:42127)
0809 02:16:45.090686 (+ 610us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:45.090700 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:45.090735 (+ 35us) server_negotiation.cc:407] Connection header received
0809 02:16:45.091347 (+ 612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:45.091362 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:45.091703 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:45.092032 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:45.092934 (+ 902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:45.092938 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:45.092949 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:16:45.092960 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:45.093015 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:16:45.093031 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:45.093206 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:45.093276 (+ 70us) server_negotiation.cc:299] Negotiation successful
0809 02:16:45.093504 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":125,"threads_started":1}
I20260809 02:16:45.094583 30527 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 10844fceb3404bf68c58829948d7431d: Neither blocks nor log segments found. Creating new log.
I20260809 02:16:45.095420 30527 log.cc:824] T 00000000000000000000000000000000 P 10844fceb3404bf68c58829948d7431d: Log is configured to *not* fsync() on all Append() calls
I20260809 02:16:45.097383 30527 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10844fceb3404bf68c58829948d7431d: No bootstrap required, opened a new log
I20260809 02:16:45.101177 30527 raft_consensus.cc:348] T 00000000000000000000000000000000 P 10844fceb3404bf68c58829948d7431d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10844fceb3404bf68c58829948d7431d" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 42127 } }
I20260809 02:16:45.101385 30527 raft_consensus.cc:374] T 00000000000000000000000000000000 P 10844fceb3404bf68c58829948d7431d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:16:45.101420 30527 raft_consensus.cc:729] T 00000000000000000000000000000000 P 10844fceb3404bf68c58829948d7431d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10844fceb3404bf68c58829948d7431d, State: Initialized, Role: FOLLOWER
I20260809 02:16:45.101769 30527 consensus_queue.cc:260] T 00000000000000000000000000000000 P 10844fceb3404bf68c58829948d7431d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10844fceb3404bf68c58829948d7431d" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 42127 } }
I20260809 02:16:45.101888 30527 raft_consensus.cc:388] T 00000000000000000000000000000000 P 10844fceb3404bf68c58829948d7431d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:16:45.101934 30527 raft_consensus.cc:482] T 00000000000000000000000000000000 P 10844fceb3404bf68c58829948d7431d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:16:45.102002 30527 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 10844fceb3404bf68c58829948d7431d [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:16:45.102877 30527 raft_consensus.cc:504] T 00000000000000000000000000000000 P 10844fceb3404bf68c58829948d7431d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10844fceb3404bf68c58829948d7431d" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 42127 } }
I20260809 02:16:45.103163 30527 leader_election.cc:302] T 00000000000000000000000000000000 P 10844fceb3404bf68c58829948d7431d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 10844fceb3404bf68c58829948d7431d; no voters:
I20260809 02:16:45.103401 30527 leader_election.cc:288] T 00000000000000000000000000000000 P 10844fceb3404bf68c58829948d7431d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:16:45.103575 30535 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 10844fceb3404bf68c58829948d7431d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:16:45.103789 30535 raft_consensus.cc:686] T 00000000000000000000000000000000 P 10844fceb3404bf68c58829948d7431d [term 1 LEADER]: Becoming Leader. State: Replica: 10844fceb3404bf68c58829948d7431d, State: Running, Role: LEADER
I20260809 02:16:45.104106 30535 consensus_queue.cc:237] T 00000000000000000000000000000000 P 10844fceb3404bf68c58829948d7431d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10844fceb3404bf68c58829948d7431d" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 42127 } }
I20260809 02:16:45.104631 30527 sys_catalog.cc:564] T 00000000000000000000000000000000 P 10844fceb3404bf68c58829948d7431d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:16:45.105775 30537 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10844fceb3404bf68c58829948d7431d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "10844fceb3404bf68c58829948d7431d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10844fceb3404bf68c58829948d7431d" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 42127 } } }
I20260809 02:16:45.105916 30537 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10844fceb3404bf68c58829948d7431d [sys.catalog]: This master's current role is: LEADER
I20260809 02:16:45.106189 30538 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10844fceb3404bf68c58829948d7431d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 10844fceb3404bf68c58829948d7431d. Latest consensus state: current_term: 1 leader_uuid: "10844fceb3404bf68c58829948d7431d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10844fceb3404bf68c58829948d7431d" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 42127 } } }
I20260809 02:16:45.106281 30538 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10844fceb3404bf68c58829948d7431d [sys.catalog]: This master's current role is: LEADER
I20260809 02:16:45.106360 30550 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:16:45.109234 30550 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:16:45.114580 30550 catalog_manager.cc:1347] Generated new cluster ID: bd2c7d967e2e40b3a94e6eb3a53ae9ea
I20260809 02:16:45.114683 30550 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:16:45.131721 30550 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:16:45.132050 30550 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:16:45.139870 30550 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 10844fceb3404bf68c58829948d7431d: Generated new TSK 0
I20260809 02:16:45.140600 30550 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:16:45.143757 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.129:0
--local_ip_for_outbound_sockets=127.15.75.129
--webserver_interface=127.15.75.129
--webserver_port=0
--tserver_master_addrs=127.15.75.190:42127
--builtin_ntp_servers=127.15.75.148:33831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 02:16:45.256475 30566 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:16:45.256757 30566 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:16:45.256851 30566 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:16:45.260838 30566 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:16:45.260878 30566 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:16:45.260979 30566 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.129
I20260809 02:16:45.265573 30566 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:33831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.75.129
--webserver_port=0
--tserver_master_addrs=127.15.75.190:42127
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30566
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.129
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:16:45.266566 30566 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:16:45.267901 30566 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:16:45.277395 30579 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:45.277493 30566 server_base.cc:1034] running on GCE node
W20260809 02:16:45.277431 30575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:45.277447 30576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:45.278046 30566 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:16:45.278652 30566 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:16:45.279838 30566 hybrid_clock.cc:648] HybridClock initialized: now 1786241805279824 us; error 25 us; skew 500 ppm
I20260809 02:16:45.281701 30566 webserver.cc:460] Webserver started at http://127.15.75.129:40905/ using document root <none> and password file <none>
I20260809 02:16:45.282275 30566 fs_manager.cc:359] Metadata directory not provided
I20260809 02:16:45.282378 30566 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:16:45.282697 30566 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:16:45.284770 30566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-0/data/instance:
uuid: "db53d26964874a17a88ab20af4a55ddf"
format_stamp: "Formatted at 2026-08-09 02:16:45 on dist-test-slave-s11t"
server_key: "ef25bef0fccc570ecb6d9ebffc1815dc"
server_key_iv: "690b39b4d2e96ea024e63100daef7360"
server_key_version: "encryptionkey@0"
I20260809 02:16:45.285434 30566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-0/wal/instance:
uuid: "db53d26964874a17a88ab20af4a55ddf"
format_stamp: "Formatted at 2026-08-09 02:16:45 on dist-test-slave-s11t"
server_key: "ef25bef0fccc570ecb6d9ebffc1815dc"
server_key_iv: "690b39b4d2e96ea024e63100daef7360"
server_key_version: "encryptionkey@0"
I20260809 02:16:45.289113 30566 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.002s
I20260809 02:16:45.291538 30591 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:45.292821 30566 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 02:16:45.292974 30566 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-0/wal
uuid: "db53d26964874a17a88ab20af4a55ddf"
format_stamp: "Formatted at 2026-08-09 02:16:45 on dist-test-slave-s11t"
server_key: "ef25bef0fccc570ecb6d9ebffc1815dc"
server_key_iv: "690b39b4d2e96ea024e63100daef7360"
server_key_version: "encryptionkey@0"
I20260809 02:16:45.293088 30566 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:16:45.297546 30566 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:16:45.298149 30566 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:16:45.298338 30566 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:16:45.298869 30566 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:16:45.299754 30566 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:16:45.299825 30566 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:45.299894 30566 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:16:45.299953 30566 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:45.308911 30566 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.129:39613
I20260809 02:16:45.308936 30758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.129:39613 every 8 connection(s)
I20260809 02:16:45.309768 30566 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
I20260809 02:16:45.310524 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 30566
I20260809 02:16:45.310674 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-0/wal/instance
I20260809 02:16:45.310971 15662 external_mini_cluster.cc:1428] Setting key c50f94dad6e67d24e147b495d6323ff6
I20260809 02:16:45.313153 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.130:0
--local_ip_for_outbound_sockets=127.15.75.130
--webserver_interface=127.15.75.130
--webserver_port=0
--tserver_master_addrs=127.15.75.190:42127
--builtin_ntp_servers=127.15.75.148:33831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260809 02:16:45.317041 30533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:45.311580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.129:37853 (local address 127.15.75.190:42127)
0809 02:16:45.311723 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:45.311731 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:45.313046 (+ 1315us) server_negotiation.cc:407] Connection header received
0809 02:16:45.315077 (+ 2031us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:45.315083 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:45.315135 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:45.315196 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:45.316500 (+ 1304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:45.316504 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:45.316507 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:16:45.316518 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:45.316540 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:16:45.316555 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:45.316647 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:45.316855 (+ 208us) server_negotiation.cc:299] Negotiation successful
0809 02:16:45.316900 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 02:16:45.317934 30765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:45.312035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:42127 (local address 127.15.75.129:37853)
0809 02:16:45.312882 (+ 847us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:45.312941 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:45.314811 (+ 1870us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:45.315531 (+ 720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:45.315537 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:45.315619 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:45.316232 (+ 613us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:16:45.316234 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:45.316274 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:16:45.316342 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:16:45.316646 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:16:45.316659 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:16:45.316667 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 02:16:45.316887 (+ 220us) client_negotiation.cc:240] Negotiation successful
0809 02:16:45.317125 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":165,"threads_started":1}
I20260809 02:16:45.319141 30759 heartbeater.cc:347] Connected to a master server at 127.15.75.190:42127
I20260809 02:16:45.319408 30759 heartbeater.cc:464] Registering TS with master...
I20260809 02:16:45.319903 30759 heartbeater.cc:511] Master 127.15.75.190:42127 requested a full tablet report, sending...
I20260809 02:16:45.321707 30469 ts_manager.cc:194] Registered new tserver with Master: db53d26964874a17a88ab20af4a55ddf (127.15.75.129:39613)
W20260809 02:16:45.425535 30766 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:16:45.425766 30766 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:16:45.425804 30766 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:16:45.429701 30766 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:16:45.429749 30766 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:16:45.429801 30766 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.130
I20260809 02:16:45.434315 30766 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:33831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.75.130
--webserver_port=0
--tserver_master_addrs=127.15.75.190:42127
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30766
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.130
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:16:45.435250 30766 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:16:45.436550 30766 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:16:45.446856 30774 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:45.446867 30771 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:45.446861 30772 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:45.447117 30766 server_base.cc:1034] running on GCE node
I20260809 02:16:45.447655 30766 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:16:45.448307 30766 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:16:45.449483 30766 hybrid_clock.cc:648] HybridClock initialized: now 1786241805449470 us; error 24 us; skew 500 ppm
I20260809 02:16:45.451215 30766 webserver.cc:460] Webserver started at http://127.15.75.130:43485/ using document root <none> and password file <none>
I20260809 02:16:45.451794 30766 fs_manager.cc:359] Metadata directory not provided
I20260809 02:16:45.451857 30766 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:16:45.452091 30766 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:16:45.454203 30766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-1/data/instance:
uuid: "09dc739e0ac74ec68c90c2344eb45c53"
format_stamp: "Formatted at 2026-08-09 02:16:45 on dist-test-slave-s11t"
server_key: "eaf132d44c6e34e36037acb1dfb64de5"
server_key_iv: "0688c14931fe1ed18f2860bfe1ba0214"
server_key_version: "encryptionkey@0"
I20260809 02:16:45.455335 30766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-1/wal/instance:
uuid: "09dc739e0ac74ec68c90c2344eb45c53"
format_stamp: "Formatted at 2026-08-09 02:16:45 on dist-test-slave-s11t"
server_key: "eaf132d44c6e34e36037acb1dfb64de5"
server_key_iv: "0688c14931fe1ed18f2860bfe1ba0214"
server_key_version: "encryptionkey@0"
I20260809 02:16:45.459090 30766 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 02:16:45.461602 30783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:45.462849 30766 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 02:16:45.462970 30766 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-1/wal
uuid: "09dc739e0ac74ec68c90c2344eb45c53"
format_stamp: "Formatted at 2026-08-09 02:16:45 on dist-test-slave-s11t"
server_key: "eaf132d44c6e34e36037acb1dfb64de5"
server_key_iv: "0688c14931fe1ed18f2860bfe1ba0214"
server_key_version: "encryptionkey@0"
I20260809 02:16:45.463074 30766 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:16:45.467697 30766 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:16:45.468284 30766 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:16:45.468472 30766 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:16:45.469092 30766 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:16:45.470052 30766 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:16:45.470100 30766 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:45.470166 30766 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:16:45.470206 30766 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:45.478976 30766 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.130:45987
I20260809 02:16:45.479000 30973 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.130:45987 every 8 connection(s)
I20260809 02:16:45.479818 30766 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
I20260809 02:16:45.480106 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 30766
I20260809 02:16:45.480235 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-1/wal/instance
I20260809 02:16:45.480584 15662 external_mini_cluster.cc:1428] Setting key c0db18fe66441ec94a1d869bf59c67cf
I20260809 02:16:45.484382 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.131:0
--local_ip_for_outbound_sockets=127.15.75.131
--webserver_interface=127.15.75.131
--webserver_port=0
--tserver_master_addrs=127.15.75.190:42127
--builtin_ntp_servers=127.15.75.148:33831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260809 02:16:45.486562 30533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:45.481694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.130:52183 (local address 127.15.75.190:42127)
0809 02:16:45.481860 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:45.481865 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:45.482912 (+ 1047us) server_negotiation.cc:407] Connection header received
0809 02:16:45.484440 (+ 1528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:45.484446 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:45.484491 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:45.484582 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:45.486008 (+ 1426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:45.486012 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:45.486013 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:16:45.486025 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:45.486046 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:16:45.486054 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:45.486127 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:45.486336 (+ 209us) server_negotiation.cc:299] Negotiation successful
0809 02:16:45.486386 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 02:16:45.487375 30978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:45.482134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:42127 (local address 127.15.75.130:52183)
0809 02:16:45.482777 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:45.482813 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:45.484236 (+ 1423us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:45.484873 (+ 637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:45.484884 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:45.484999 (+ 115us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:45.485683 (+ 684us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:16:45.485688 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:45.485743 (+ 55us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:16:45.485847 (+ 104us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:16:45.486143 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:16:45.486154 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:16:45.486163 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 02:16:45.486363 (+ 200us) client_negotiation.cc:240] Negotiation successful
0809 02:16:45.486635 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":118,"threads_started":1}
I20260809 02:16:45.488474 30974 heartbeater.cc:347] Connected to a master server at 127.15.75.190:42127
I20260809 02:16:45.488727 30974 heartbeater.cc:464] Registering TS with master...
I20260809 02:16:45.489214 30974 heartbeater.cc:511] Master 127.15.75.190:42127 requested a full tablet report, sending...
I20260809 02:16:45.490386 30469 ts_manager.cc:194] Registered new tserver with Master: 09dc739e0ac74ec68c90c2344eb45c53 (127.15.75.130:45987)
W20260809 02:16:45.596419 30979 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:16:45.596668 30979 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:16:45.596788 30979 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:16:45.600826 30979 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:16:45.600874 30979 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:16:45.600956 30979 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.131
I20260809 02:16:45.605890 30979 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:33831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.75.131
--webserver_port=0
--tserver_master_addrs=127.15.75.190:42127
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30979
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.131
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:16:45.606887 30979 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:16:45.608207 30979 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:16:45.618170 30989 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:45.618201 30979 server_base.cc:1034] running on GCE node
W20260809 02:16:45.618203 30992 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:45.618281 30988 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:45.618783 30979 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:16:45.619377 30979 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:16:45.620540 30979 hybrid_clock.cc:648] HybridClock initialized: now 1786241805620524 us; error 27 us; skew 500 ppm
I20260809 02:16:45.622287 30979 webserver.cc:460] Webserver started at http://127.15.75.131:36265/ using document root <none> and password file <none>
I20260809 02:16:45.622852 30979 fs_manager.cc:359] Metadata directory not provided
I20260809 02:16:45.622942 30979 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:16:45.623179 30979 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:16:45.625213 30979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-2/data/instance:
uuid: "bc151415ea6047e1a7f0daed864ab1ba"
format_stamp: "Formatted at 2026-08-09 02:16:45 on dist-test-slave-s11t"
server_key: "5f1d9779bbfd3eeb1e4d37e33c160cde"
server_key_iv: "d207562ce5d683ec04b495e3f071e3be"
server_key_version: "encryptionkey@0"
I20260809 02:16:45.625762 30979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-2/wal/instance:
uuid: "bc151415ea6047e1a7f0daed864ab1ba"
format_stamp: "Formatted at 2026-08-09 02:16:45 on dist-test-slave-s11t"
server_key: "5f1d9779bbfd3eeb1e4d37e33c160cde"
server_key_iv: "d207562ce5d683ec04b495e3f071e3be"
server_key_version: "encryptionkey@0"
I20260809 02:16:45.629348 30979 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 02:16:45.631734 31004 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:45.632973 30979 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 02:16:45.633088 30979 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-2/wal
uuid: "bc151415ea6047e1a7f0daed864ab1ba"
format_stamp: "Formatted at 2026-08-09 02:16:45 on dist-test-slave-s11t"
server_key: "5f1d9779bbfd3eeb1e4d37e33c160cde"
server_key_iv: "d207562ce5d683ec04b495e3f071e3be"
server_key_version: "encryptionkey@0"
I20260809 02:16:45.633193 30979 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:16:45.637681 30979 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:16:45.638284 30979 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:16:45.638465 30979 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:16:45.639007 30979 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:16:45.640021 30979 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:16:45.640089 30979 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:45.640165 30979 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:16:45.640203 30979 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:45.649268 30979 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.131:46677
I20260809 02:16:45.649303 31170 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.131:46677 every 8 connection(s)
I20260809 02:16:45.650167 30979 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
I20260809 02:16:45.651731 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 30979
I20260809 02:16:45.651813 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241741788589-15662-0/minicluster-data/ts-2/wal/instance
I20260809 02:16:45.652141 15662 external_mini_cluster.cc:1428] Setting key 7537bd5391d714c134671dc9163c26f4
I20260809 02:16:45.656797 30533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:45.652056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.131:37189 (local address 127.15.75.190:42127)
0809 02:16:45.652207 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:45.652213 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:45.653688 (+ 1475us) server_negotiation.cc:407] Connection header received
0809 02:16:45.655145 (+ 1457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:45.655148 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:45.655192 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:45.655270 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:45.656273 (+ 1003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:45.656276 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:45.656279 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:16:45.656291 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:45.656314 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:16:45.656330 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:45.656421 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:45.656581 (+ 160us) server_negotiation.cc:299] Negotiation successful
0809 02:16:45.656624 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 02:16:45.657600 31177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:45.652548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:42127 (local address 127.15.75.131:37189)
0809 02:16:45.653554 (+ 1006us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:45.653608 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:45.654946 (+ 1338us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:45.655437 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:45.655443 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:45.655528 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:45.656027 (+ 499us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:16:45.656028 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:45.656067 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:16:45.656132 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:16:45.656416 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:16:45.656426 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:16:45.656434 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 02:16:45.656611 (+ 177us) client_negotiation.cc:240] Negotiation successful
0809 02:16:45.656872 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":125,"threads_started":1}
I20260809 02:16:45.658619 31171 heartbeater.cc:347] Connected to a master server at 127.15.75.190:42127
I20260809 02:16:45.658823 31171 heartbeater.cc:464] Registering TS with master...
I20260809 02:16:45.659183 31171 heartbeater.cc:511] Master 127.15.75.190:42127 requested a full tablet report, sending...
I20260809 02:16:45.660158 30469 ts_manager.cc:194] Registered new tserver with Master: bc151415ea6047e1a7f0daed864ab1ba (127.15.75.131:46677)
I20260809 02:16:45.666363 15662 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 02:16:45.674490 30533 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:16:45.668960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44204 (local address 127.15.75.190:42127)
0809 02:16:45.669084 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:45.669089 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:45.673738 (+ 4649us) server_negotiation.cc:407] Connection header received
0809 02:16:45.673899 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:45.673904 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:45.673947 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:45.674015 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:45.674334 (+ 319us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44204: BlockingRecv error: recv got EOF from 127.0.0.1:44204 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 02:16:45.675246 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 30566
I20260809 02:16:45.691525 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 30766
I20260809 02:16:45.701357 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 30979
I20260809 02:16:45.709952 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 30419
2026-08-09T02:16:45Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (821 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:16:45 dist-test-slave-s11t krb5kdc[31188](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:16:45 dist-test-slave-s11t krb5kdc[31188](info): set up 2 sockets
Aug 09 02:16:45 dist-test-slave-s11t krb5kdc[31188](info): commencing operation
krb5kdc: starting...
W20260809 02:16:46.956485 15662 mini_kdc.cc:121] Time spent starting KDC: real 1.222s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Aug 09 02:16:47 dist-test-slave-s11t krb5kdc[31188](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241807, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:16:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:16:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.75.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab.
Entry for principal kudu/127.15.75.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab.
I20260809 02:16:47.083580 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:40319
--webserver_interface=127.15.75.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab
--principal=kudu/127.15.75.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:35431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.75.190:40319
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:16:47.196580 31206 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:16:47.196898 31206 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:16:47.196978 31206 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:16:47.200980 31206 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:16:47.201067 31206 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:16:47.201105 31206 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:16:47.201162 31206 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:16:47.201195 31206 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:16:47.206377 31206 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:35431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.75.190:40319
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab
--principal=kudu/127.15.75.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:40319
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.75.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31206
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:16:47.207412 31206 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:16:47.208446 31206 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:16:47.217196 31215 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:47.217213 31221 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:47.217252 31216 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:47.217370 31206 server_base.cc:1034] running on GCE node
I20260809 02:16:47.217957 31206 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:16:47.219074 31206 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:16:47.220278 31206 hybrid_clock.cc:648] HybridClock initialized: now 1786241807220242 us; error 47 us; skew 500 ppm
Aug 09 02:16:47 dist-test-slave-s11t krb5kdc[31188](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241807, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:16:47.223150 31206 init.cc:377] Logged in from keytab as kudu/127.15.75.190@KRBTEST.COM (short username kudu)
I20260809 02:16:47.224110 31206 webserver.cc:460] Webserver started at http://127.15.75.190:37437/ using document root <none> and password file <none>
I20260809 02:16:47.224776 31206 fs_manager.cc:359] Metadata directory not provided
I20260809 02:16:47.224875 31206 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:16:47.225122 31206 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:16:47.227192 31206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/minicluster-data/master-0/data/instance:
uuid: "74715eccb6c343149c3ee8d054201318"
format_stamp: "Formatted at 2026-08-09 02:16:47 on dist-test-slave-s11t"
server_key: "b8c313f7bb541508825a2c2ff8c129b8"
server_key_iv: "35e053a58e8f0a61aeef1345427f4c47"
server_key_version: "encryptionkey@0"
I20260809 02:16:47.227773 31206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/minicluster-data/master-0/wal/instance:
uuid: "74715eccb6c343149c3ee8d054201318"
format_stamp: "Formatted at 2026-08-09 02:16:47 on dist-test-slave-s11t"
server_key: "b8c313f7bb541508825a2c2ff8c129b8"
server_key_iv: "35e053a58e8f0a61aeef1345427f4c47"
server_key_version: "encryptionkey@0"
I20260809 02:16:47.231491 31206 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260809 02:16:47.233942 31232 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:47.235173 31206 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 02:16:47.235320 31206 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/minicluster-data/master-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/minicluster-data/master-0/wal
uuid: "74715eccb6c343149c3ee8d054201318"
format_stamp: "Formatted at 2026-08-09 02:16:47 on dist-test-slave-s11t"
server_key: "b8c313f7bb541508825a2c2ff8c129b8"
server_key_iv: "35e053a58e8f0a61aeef1345427f4c47"
server_key_version: "encryptionkey@0"
I20260809 02:16:47.235430 31206 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:16:47.273355 31206 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:16:47.276373 31206 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:16:47.276592 31206 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:16:47.284009 31206 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.190:40319
I20260809 02:16:47.284317 31318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.190:40319 every 8 connection(s)
I20260809 02:16:47.285045 31206 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
I20260809 02:16:47.287833 31320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:16:47.290796 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 31206
I20260809 02:16:47.290943 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/minicluster-data/master-0/wal/instance
I20260809 02:16:47.291215 15662 external_mini_cluster.cc:1428] Setting key 92e939dd917e3f22a8700605d2eb0392
I20260809 02:16:47.294005 31320 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74715eccb6c343149c3ee8d054201318: Bootstrap starting.
I20260809 02:16:47.296680 31320 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 74715eccb6c343149c3ee8d054201318: Neither blocks nor log segments found. Creating new log.
I20260809 02:16:47.297605 31320 log.cc:824] T 00000000000000000000000000000000 P 74715eccb6c343149c3ee8d054201318: Log is configured to *not* fsync() on all Append() calls
I20260809 02:16:47.300016 31320 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74715eccb6c343149c3ee8d054201318: No bootstrap required, opened a new log
Aug 09 02:16:47 dist-test-slave-s11t krb5kdc[31188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241807, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:16:47.305145 31320 raft_consensus.cc:348] T 00000000000000000000000000000000 P 74715eccb6c343149c3ee8d054201318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74715eccb6c343149c3ee8d054201318" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 40319 } }
I20260809 02:16:47.305433 31320 raft_consensus.cc:374] T 00000000000000000000000000000000 P 74715eccb6c343149c3ee8d054201318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:16:47.305495 31320 raft_consensus.cc:729] T 00000000000000000000000000000000 P 74715eccb6c343149c3ee8d054201318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74715eccb6c343149c3ee8d054201318, State: Initialized, Role: FOLLOWER
I20260809 02:16:47.305987 31320 consensus_queue.cc:260] T 00000000000000000000000000000000 P 74715eccb6c343149c3ee8d054201318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74715eccb6c343149c3ee8d054201318" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 40319 } }
I20260809 02:16:47.306142 31320 raft_consensus.cc:388] T 00000000000000000000000000000000 P 74715eccb6c343149c3ee8d054201318 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:16:47.306207 31320 raft_consensus.cc:482] T 00000000000000000000000000000000 P 74715eccb6c343149c3ee8d054201318 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:16:47.306300 31320 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 74715eccb6c343149c3ee8d054201318 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:16:47.306444 31325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:47.292561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60534 (local address 127.15.75.190:40319)
0809 02:16:47.293005 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:47.293020 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:47.293055 (+ 35us) server_negotiation.cc:407] Connection header received
0809 02:16:47.293664 (+ 609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:47.293680 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:47.294004 (+ 324us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:47.294323 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:47.295217 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:47.296238 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:47.297467 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:47.298023 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:47.298206 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:47.301145 (+ 2939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:47.301152 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:47.301169 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:47.301227 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:47.303239 (+ 2012us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:47.303694 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:47.303701 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:47.303707 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:47.303855 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:47.304141 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:47.304144 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:47.304146 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:47.304716 (+ 570us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:47.304936 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:47.305138 (+ 202us) server_negotiation.cc:299] Negotiation successful
0809 02:16:47.305456 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":157,"threads_started":1}
I20260809 02:16:47.307298 31320 raft_consensus.cc:504] T 00000000000000000000000000000000 P 74715eccb6c343149c3ee8d054201318 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74715eccb6c343149c3ee8d054201318" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 40319 } }
I20260809 02:16:47.307617 31320 leader_election.cc:302] T 00000000000000000000000000000000 P 74715eccb6c343149c3ee8d054201318 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 74715eccb6c343149c3ee8d054201318; no voters:
I20260809 02:16:47.307896 31320 leader_election.cc:288] T 00000000000000000000000000000000 P 74715eccb6c343149c3ee8d054201318 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:16:47.307986 31328 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 74715eccb6c343149c3ee8d054201318 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:16:47.308193 31328 raft_consensus.cc:686] T 00000000000000000000000000000000 P 74715eccb6c343149c3ee8d054201318 [term 1 LEADER]: Becoming Leader. State: Replica: 74715eccb6c343149c3ee8d054201318, State: Running, Role: LEADER
I20260809 02:16:47.308507 31328 consensus_queue.cc:237] T 00000000000000000000000000000000 P 74715eccb6c343149c3ee8d054201318 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74715eccb6c343149c3ee8d054201318" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 40319 } }
I20260809 02:16:47.309217 31320 sys_catalog.cc:564] T 00000000000000000000000000000000 P 74715eccb6c343149c3ee8d054201318 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:16:47.310410 31330 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74715eccb6c343149c3ee8d054201318 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 74715eccb6c343149c3ee8d054201318. Latest consensus state: current_term: 1 leader_uuid: "74715eccb6c343149c3ee8d054201318" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74715eccb6c343149c3ee8d054201318" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 40319 } } }
I20260809 02:16:47.310420 31329 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74715eccb6c343149c3ee8d054201318 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "74715eccb6c343149c3ee8d054201318" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74715eccb6c343149c3ee8d054201318" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 40319 } } }
I20260809 02:16:47.310576 31330 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74715eccb6c343149c3ee8d054201318 [sys.catalog]: This master's current role is: LEADER
I20260809 02:16:47.310578 31329 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74715eccb6c343149c3ee8d054201318 [sys.catalog]: This master's current role is: LEADER
I20260809 02:16:47.311028 31338 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:16:47.314163 31338 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:16:47.319707 31338 catalog_manager.cc:1347] Generated new cluster ID: 5f2e3fddff9e4c069ea0b1fa5913f522
I20260809 02:16:47.319774 31338 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:16:47.337570 31338 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:16:47.338845 31338 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:16:47.360625 31338 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 74715eccb6c343149c3ee8d054201318: Generated new TSK 0
I20260809 02:16:47.361438 31338 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.75.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab.
Entry for principal kudu/127.15.75.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab.
I20260809 02:16:47.400206 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.129:0
--local_ip_for_outbound_sockets=127.15.75.129
--webserver_interface=127.15.75.129
--webserver_port=0
--tserver_master_addrs=127.15.75.190:40319
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab
--principal=kudu/127.15.75.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:35431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:16:47.513517 31355 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:16:47.513792 31355 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:16:47.513863 31355 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:16:47.517830 31355 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:16:47.517894 31355 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:16:47.517999 31355 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.129
I20260809 02:16:47.523283 31355 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:35431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab
--principal=kudu/127.15.75.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.75.129
--webserver_port=0
--tserver_master_addrs=127.15.75.190:40319
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31355
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.129
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:16:47.524401 31355 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:16:47.525502 31355 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:16:47.535753 31364 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:47.535801 31355 server_base.cc:1034] running on GCE node
W20260809 02:16:47.535790 31370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:47.535792 31366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:47.536437 31355 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:16:47.537122 31355 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:16:47.538290 31355 hybrid_clock.cc:648] HybridClock initialized: now 1786241807538261 us; error 39 us; skew 500 ppm
Aug 09 02:16:47 dist-test-slave-s11t krb5kdc[31188](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241807, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:16:47.541268 31355 init.cc:377] Logged in from keytab as kudu/127.15.75.129@KRBTEST.COM (short username kudu)
I20260809 02:16:47.542189 31355 webserver.cc:460] Webserver started at http://127.15.75.129:34011/ using document root <none> and password file <none>
I20260809 02:16:47.542776 31355 fs_manager.cc:359] Metadata directory not provided
I20260809 02:16:47.542840 31355 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:16:47.543059 31355 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:16:47.545172 31355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/minicluster-data/ts-0/data/instance:
uuid: "d872f435a0174e0a8995d2305aa18a8f"
format_stamp: "Formatted at 2026-08-09 02:16:47 on dist-test-slave-s11t"
server_key: "a7699d78df6a7e5f8c1bb10551d50a10"
server_key_iv: "defd8f3b6be5cdd2686b382b6b5c7cb7"
server_key_version: "encryptionkey@0"
I20260809 02:16:47.545738 31355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/minicluster-data/ts-0/wal/instance:
uuid: "d872f435a0174e0a8995d2305aa18a8f"
format_stamp: "Formatted at 2026-08-09 02:16:47 on dist-test-slave-s11t"
server_key: "a7699d78df6a7e5f8c1bb10551d50a10"
server_key_iv: "defd8f3b6be5cdd2686b382b6b5c7cb7"
server_key_version: "encryptionkey@0"
I20260809 02:16:47.549479 31355 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 02:16:47.551898 31381 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:47.553223 31355 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 02:16:47.553355 31355 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/minicluster-data/ts-0/wal
uuid: "d872f435a0174e0a8995d2305aa18a8f"
format_stamp: "Formatted at 2026-08-09 02:16:47 on dist-test-slave-s11t"
server_key: "a7699d78df6a7e5f8c1bb10551d50a10"
server_key_iv: "defd8f3b6be5cdd2686b382b6b5c7cb7"
server_key_version: "encryptionkey@0"
I20260809 02:16:47.553493 31355 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:16:47.567703 31355 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:16:47.570773 31355 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:16:47.570990 31355 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:16:47.571606 31355 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:16:47.572459 31355 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:16:47.572530 31355 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:47.572602 31355 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:16:47.572642 31355 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:47.581593 31355 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.129:44729
I20260809 02:16:47.581624 31554 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.129:44729 every 8 connection(s)
I20260809 02:16:47.582485 31355 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
I20260809 02:16:47.587054 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 31355
I20260809 02:16:47.587185 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/minicluster-data/ts-0/wal/instance
I20260809 02:16:47.587505 15662 external_mini_cluster.cc:1428] Setting key 8d43b752f5405475a6319b2f7bff203a
Aug 09 02:16:47 dist-test-slave-s11t krb5kdc[31188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241807, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.129@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:16:47.598063 31325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:47.584486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.129:60517 (local address 127.15.75.190:40319)
0809 02:16:47.584658 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:47.584663 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:47.585323 (+ 660us) server_negotiation.cc:407] Connection header received
0809 02:16:47.586686 (+ 1363us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:47.586689 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:47.586736 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:47.586810 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:47.588406 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:47.589183 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:47.590899 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:47.591164 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:47.591289 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:47.594095 (+ 2806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:47.594100 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:47.594103 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:47.594134 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:47.595762 (+ 1628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:47.596424 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:47.596427 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:47.596428 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:47.596502 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:47.596935 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:47.596938 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:47.596939 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:47.597113 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:47.597189 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:47.597837 (+ 648us) server_negotiation.cc:299] Negotiation successful
0809 02:16:47.597944 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
WARNING: no policy specified for kudu/127.15.75.130@KRBTEST.COM; defaulting to no policy
I20260809 02:16:47.599301 31557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:47.584658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:40319 (local address 127.15.75.129:60517)
0809 02:16:47.585164 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:47.585207 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:47.586495 (+ 1288us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:47.587016 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:47.587025 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:47.587558 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:47.588219 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:47.588240 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:47.589325 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:47.589328 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:47.590776 (+ 1448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:47.590790 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:47.591354 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:47.591358 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:47.591519 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:47.592145 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:47.592163 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:47.593919 (+ 1756us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:16:47.595882 (+ 1963us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:47.595892 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:47.595910 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:47.596316 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:47.596595 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:47.596601 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:47.596603 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:47.596832 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:47.597222 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:16:47.597229 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:16:47.597567 (+ 338us) client_negotiation.cc:769] Sending connection context
0809 02:16:47.597791 (+ 224us) client_negotiation.cc:240] Negotiation successful
0809 02:16:47.598154 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":111,"threads_started":1}
I20260809 02:16:47.600673 31555 heartbeater.cc:347] Connected to a master server at 127.15.75.190:40319
I20260809 02:16:47.600996 31555 heartbeater.cc:464] Registering TS with master...
I20260809 02:16:47.601550 31555 heartbeater.cc:511] Master 127.15.75.190:40319 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.130@KRBTEST.COM" created.
I20260809 02:16:47.603211 31262 ts_manager.cc:194] Registered new tserver with Master: d872f435a0174e0a8995d2305aa18a8f (127.15.75.129:44729)
I20260809 02:16:47.604904 31262 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.75.129@KRBTEST.COM'} at 127.15.75.129:60517
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab.
Entry for principal kudu/127.15.75.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab.
I20260809 02:16:47.619369 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.130:0
--local_ip_for_outbound_sockets=127.15.75.130
--webserver_interface=127.15.75.130
--webserver_port=0
--tserver_master_addrs=127.15.75.190:40319
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab
--principal=kudu/127.15.75.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:35431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:16:47.735507 31561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:16:47.735750 31561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:16:47.735793 31561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:16:47.739931 31561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:16:47.739993 31561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:16:47.740093 31561 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.130
I20260809 02:16:47.745734 31561 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:35431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab
--principal=kudu/127.15.75.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.75.130
--webserver_port=0
--tserver_master_addrs=127.15.75.190:40319
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31561
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.130
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:16:47.746742 31561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:16:47.747705 31561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:16:47.757349 31572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:47.757368 31575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:47.757381 31571 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:47.757606 31561 server_base.cc:1034] running on GCE node
I20260809 02:16:47.758054 31561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:16:47.758701 31561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:16:47.759899 31561 hybrid_clock.cc:648] HybridClock initialized: now 1786241807759885 us; error 32 us; skew 500 ppm
Aug 09 02:16:47 dist-test-slave-s11t krb5kdc[31188](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241807, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:16:47.762709 31561 init.cc:377] Logged in from keytab as kudu/127.15.75.130@KRBTEST.COM (short username kudu)
I20260809 02:16:47.763607 31561 webserver.cc:460] Webserver started at http://127.15.75.130:35483/ using document root <none> and password file <none>
I20260809 02:16:47.764186 31561 fs_manager.cc:359] Metadata directory not provided
I20260809 02:16:47.764279 31561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:16:47.764523 31561 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:16:47.766608 31561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/minicluster-data/ts-1/data/instance:
uuid: "3ca04362b50c4480b5164a01d12f78f9"
format_stamp: "Formatted at 2026-08-09 02:16:47 on dist-test-slave-s11t"
server_key: "1b7f520d34ec3134953b0a8c787abd74"
server_key_iv: "028ba877c5a3de69170111a359ebec7d"
server_key_version: "encryptionkey@0"
I20260809 02:16:47.767187 31561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/minicluster-data/ts-1/wal/instance:
uuid: "3ca04362b50c4480b5164a01d12f78f9"
format_stamp: "Formatted at 2026-08-09 02:16:47 on dist-test-slave-s11t"
server_key: "1b7f520d34ec3134953b0a8c787abd74"
server_key_iv: "028ba877c5a3de69170111a359ebec7d"
server_key_version: "encryptionkey@0"
I20260809 02:16:47.770923 31561 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 02:16:47.774111 31587 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:47.775316 31561 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 02:16:47.775467 31561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/minicluster-data/ts-1/wal
uuid: "3ca04362b50c4480b5164a01d12f78f9"
format_stamp: "Formatted at 2026-08-09 02:16:47 on dist-test-slave-s11t"
server_key: "1b7f520d34ec3134953b0a8c787abd74"
server_key_iv: "028ba877c5a3de69170111a359ebec7d"
server_key_version: "encryptionkey@0"
I20260809 02:16:47.775581 31561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:16:47.792153 31561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:16:47.795445 31561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:16:47.795652 31561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:16:47.796386 31561 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:16:47.797290 31561 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:16:47.797361 31561 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:47.797438 31561 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:16:47.797482 31561 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:47.807257 31561 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.130:33849
I20260809 02:16:47.807293 31764 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.130:33849 every 8 connection(s)
I20260809 02:16:47.808187 31561 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
I20260809 02:16:47.816644 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 31561
I20260809 02:16:47.816794 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/minicluster-data/ts-1/wal/instance
I20260809 02:16:47.817194 15662 external_mini_cluster.cc:1428] Setting key 315578271ec61b1ebf1120a65250975e
Aug 09 02:16:47 dist-test-slave-s11t krb5kdc[31188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241807, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.130@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:16:47.823181 31325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:47.810119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.130:60721 (local address 127.15.75.190:40319)
0809 02:16:47.810286 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:47.810291 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:47.811293 (+ 1002us) server_negotiation.cc:407] Connection header received
0809 02:16:47.812773 (+ 1480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:47.812777 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:47.812826 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:47.812905 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:47.814043 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:47.814811 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:47.815888 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:47.816127 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:47.816210 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:47.819316 (+ 3106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:47.819319 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:47.819322 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:47.819346 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:47.820911 (+ 1565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:47.821515 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:47.821518 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:47.821519 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:47.821599 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:47.821991 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:47.821998 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:47.822002 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:47.822200 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:47.822327 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:47.822868 (+ 541us) server_negotiation.cc:299] Negotiation successful
0809 02:16:47.823001 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 02:16:47.823997 31768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:47.810634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:40319 (local address 127.15.75.130:60721)
0809 02:16:47.811140 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:47.811182 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:47.812531 (+ 1349us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:47.813058 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:47.813064 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:47.813402 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:47.813877 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:47.813890 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:47.814923 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:47.814925 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:47.815781 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:47.815791 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:47.816259 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:47.816261 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:47.816449 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:47.817137 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:47.817158 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:47.819136 (+ 1978us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:16:47.821075 (+ 1939us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:47.821083 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:47.821102 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:47.821427 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:47.821710 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:47.821713 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:47.821716 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:47.821879 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:47.822342 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:16:47.822351 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:16:47.822595 (+ 244us) client_negotiation.cc:769] Sending connection context
0809 02:16:47.822818 (+ 223us) client_negotiation.cc:240] Negotiation successful
0809 02:16:47.823149 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":173,"threads_started":1}
I20260809 02:16:47.825302 31766 heartbeater.cc:347] Connected to a master server at 127.15.75.190:40319
I20260809 02:16:47.825587 31766 heartbeater.cc:464] Registering TS with master...
I20260809 02:16:47.826102 31766 heartbeater.cc:511] Master 127.15.75.190:40319 requested a full tablet report, sending...
I20260809 02:16:47.827220 31262 ts_manager.cc:194] Registered new tserver with Master: 3ca04362b50c4480b5164a01d12f78f9 (127.15.75.130:33849)
I20260809 02:16:47.827844 31262 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.75.130@KRBTEST.COM'} at 127.15.75.130:60721
WARNING: no policy specified for kudu/127.15.75.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab.
Entry for principal kudu/127.15.75.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab.
I20260809 02:16:47.847178 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.131:0
--local_ip_for_outbound_sockets=127.15.75.131
--webserver_interface=127.15.75.131
--webserver_port=0
--tserver_master_addrs=127.15.75.190:40319
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab
--principal=kudu/127.15.75.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:35431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:16:47.960880 31775 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:16:47.961110 31775 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:16:47.961182 31775 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:16:47.965255 31775 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:16:47.965307 31775 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:16:47.965400 31775 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.131
I20260809 02:16:47.970511 31775 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:35431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab
--principal=kudu/127.15.75.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.75.131
--webserver_port=0
--tserver_master_addrs=127.15.75.190:40319
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31775
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.131
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:16:47.971511 31775 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:16:47.972565 31775 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:16:47.982610 31787 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:47.982751 31783 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:47.982961 31775 server_base.cc:1034] running on GCE node
W20260809 02:16:47.982610 31781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:47.983456 31775 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:16:47.984069 31775 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:16:47.985249 31775 hybrid_clock.cc:648] HybridClock initialized: now 1786241807985217 us; error 46 us; skew 500 ppm
Aug 09 02:16:47 dist-test-slave-s11t krb5kdc[31188](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241807, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:16:47.987927 31775 init.cc:377] Logged in from keytab as kudu/127.15.75.131@KRBTEST.COM (short username kudu)
I20260809 02:16:47.988758 31775 webserver.cc:460] Webserver started at http://127.15.75.131:40839/ using document root <none> and password file <none>
I20260809 02:16:47.989269 31775 fs_manager.cc:359] Metadata directory not provided
I20260809 02:16:47.989326 31775 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:16:47.989498 31775 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:16:47.991485 31775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/minicluster-data/ts-2/data/instance:
uuid: "285ebddb9b174e1cbfa3a328933a8cf5"
format_stamp: "Formatted at 2026-08-09 02:16:47 on dist-test-slave-s11t"
server_key: "325e54bb578fe8801906f9d29bc29571"
server_key_iv: "59db2021d44c813a27afc72eb88ebd8a"
server_key_version: "encryptionkey@0"
I20260809 02:16:47.991982 31775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/minicluster-data/ts-2/wal/instance:
uuid: "285ebddb9b174e1cbfa3a328933a8cf5"
format_stamp: "Formatted at 2026-08-09 02:16:47 on dist-test-slave-s11t"
server_key: "325e54bb578fe8801906f9d29bc29571"
server_key_iv: "59db2021d44c813a27afc72eb88ebd8a"
server_key_version: "encryptionkey@0"
I20260809 02:16:47.995520 31775 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260809 02:16:47.997974 31798 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:47.999138 31775 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 02:16:47.999230 31775 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/minicluster-data/ts-2/wal
uuid: "285ebddb9b174e1cbfa3a328933a8cf5"
format_stamp: "Formatted at 2026-08-09 02:16:47 on dist-test-slave-s11t"
server_key: "325e54bb578fe8801906f9d29bc29571"
server_key_iv: "59db2021d44c813a27afc72eb88ebd8a"
server_key_version: "encryptionkey@0"
I20260809 02:16:47.999307 31775 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:16:48.019655 31775 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:16:48.022621 31775 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:16:48.022784 31775 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:16:48.023305 31775 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:16:48.024063 31775 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:16:48.024109 31775 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:48.024148 31775 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:16:48.024164 31775 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:48.033443 31775 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.131:40923
I20260809 02:16:48.033473 31968 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.131:40923 every 8 connection(s)
I20260809 02:16:48.034277 31775 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
I20260809 02:16:48.043677 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 31775
Aug 09 02:16:48 dist-test-slave-s11t krb5kdc[31188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241807, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.131@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:16:48.043826 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/minicluster-data/ts-2/wal/instance
I20260809 02:16:48.044317 15662 external_mini_cluster.cc:1428] Setting key 18747e917da5c2aa332cd3f8b1e8bf5b
I20260809 02:16:48.048444 31325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:48.035995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.131:38549 (local address 127.15.75.190:40319)
0809 02:16:48.036139 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:48.036143 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:48.036991 (+ 848us) server_negotiation.cc:407] Connection header received
0809 02:16:48.038491 (+ 1500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:48.038494 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:48.038556 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:48.038668 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:48.039825 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:48.040366 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:48.041569 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:48.041791 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:48.041907 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:48.044572 (+ 2665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:48.044575 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:48.044577 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:48.044602 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:48.046460 (+ 1858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:48.047119 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:48.047122 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:48.047123 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:48.047202 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:48.047528 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:48.047530 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:48.047531 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:48.047703 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:48.047780 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:48.048226 (+ 446us) server_negotiation.cc:299] Negotiation successful
0809 02:16:48.048325 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 02:16:48.049285 31973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:48.036345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:40319 (local address 127.15.75.131:38549)
0809 02:16:48.036847 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:48.036890 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:48.038257 (+ 1367us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:48.038865 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:48.038871 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:48.039211 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:48.039669 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:48.039682 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:48.040496 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:48.040500 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:48.041417 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:48.041426 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:48.041946 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:48.041951 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:48.042118 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:48.042723 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:48.042739 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:48.044399 (+ 1660us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:16:48.046634 (+ 2235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:48.046643 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:48.046660 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:48.047026 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:48.047288 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:48.047290 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:48.047292 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:48.047446 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:48.047795 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:16:48.047800 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:16:48.048024 (+ 224us) client_negotiation.cc:769] Sending connection context
0809 02:16:48.048180 (+ 156us) client_negotiation.cc:240] Negotiation successful
0809 02:16:48.048435 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":114,"threads_started":1}
I20260809 02:16:48.050393 31970 heartbeater.cc:347] Connected to a master server at 127.15.75.190:40319
I20260809 02:16:48.050597 31970 heartbeater.cc:464] Registering TS with master...
I20260809 02:16:48.051035 31970 heartbeater.cc:511] Master 127.15.75.190:40319 requested a full tablet report, sending...
I20260809 02:16:48.052034 31262 ts_manager.cc:194] Registered new tserver with Master: 285ebddb9b174e1cbfa3a328933a8cf5 (127.15.75.131:40923)
I20260809 02:16:48.052557 31262 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.75.131@KRBTEST.COM'} at 127.15.75.131:38549
I20260809 02:16:48.058411 15662 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 02:16:48.067641 31325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:48.060375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60538 (local address 127.15.75.190:40319)
0809 02:16:48.060543 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:48.060546 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:48.060632 (+ 86us) server_negotiation.cc:407] Connection header received
0809 02:16:48.060785 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:48.060787 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:48.060823 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:48.060913 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:48.061639 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:48.062148 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:48.063014 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:48.063246 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:48.063352 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:48.064649 (+ 1297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:48.064653 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:48.064656 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:48.064729 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:48.065948 (+ 1219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:48.066366 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:48.066370 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:48.066372 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:48.066458 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:48.066741 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:48.066744 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:48.066746 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:48.066969 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:48.067107 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:48.067350 (+ 243us) server_negotiation.cc:299] Negotiation successful
0809 02:16:48.067475 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 02:16:48.070490 31262 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60538:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:16:48.073441 31262 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:16:48.087787 31991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:48.081827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.131:40923 (local address 127.0.0.1:34334)
0809 02:16:48.082657 (+ 830us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:48.082671 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:48.082770 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:48.083884 (+ 1114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:48.083887 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:48.083905 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:16:48.083990 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:48.083996 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:48.084986 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:48.084990 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:48.086217 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:48.086227 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:48.087258 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:48.087262 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:48.087365 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:48.087399 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 02:16:48.087509 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 02:16:48.087623 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":771,"mutex_wait_us":6,"thread_start_us":136,"threads_started":1}
I20260809 02:16:48.088115 31989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:48.082071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49454 (local address 127.15.75.129:44729)
0809 02:16:48.082464 (+ 393us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:48.082480 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:48.082530 (+ 50us) server_negotiation.cc:407] Connection header received
0809 02:16:48.082837 (+ 307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:48.082844 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:48.083075 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:48.083234 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:16:48.083782 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:48.084918 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:48.086623 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:48.087142 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:48.087265 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:48.087410 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:48.087862 (+ 452us) server_negotiation.cc:299] Negotiation successful
0809 02:16:48.087980 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":100,"threads_started":1}
I20260809 02:16:48.088200 31987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:48.081530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56830 (local address 127.15.75.130:33849)
0809 02:16:48.082498 (+ 968us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:48.082510 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:48.082832 (+ 322us) server_negotiation.cc:407] Connection header received
0809 02:16:48.082976 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:48.082981 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:48.083154 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:48.083324 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:16:48.083641 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:48.084877 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:48.086596 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:48.087363 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:48.087479 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:48.087618 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:48.087804 (+ 186us) server_negotiation.cc:299] Negotiation successful
0809 02:16:48.088049 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":871,"thread_start_us":160,"threads_started":1}
I20260809 02:16:48.088243 31990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:48.082062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34334 (local address 127.15.75.131:40923)
0809 02:16:48.083073 (+ 1011us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:48.083090 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:48.083110 (+ 20us) server_negotiation.cc:407] Connection header received
0809 02:16:48.083594 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:48.083602 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:48.083756 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:48.083916 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:16:48.084087 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:48.084855 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:48.086344 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:48.087120 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:48.087239 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:48.087384 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:48.087997 (+ 613us) server_negotiation.cc:299] Negotiation successful
0809 02:16:48.088081 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":915,"thread_start_us":151,"threads_started":1}
I20260809 02:16:48.087971 31988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:48.081858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.130:33849 (local address 127.0.0.1:56830)
0809 02:16:48.082732 (+ 874us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:48.082746 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:48.082837 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:48.083323 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:48.083326 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:48.083343 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:16:48.083497 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:48.083524 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:48.085055 (+ 1531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:48.085060 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:48.086470 (+ 1410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:48.086482 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:48.087480 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:48.087483 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:48.087630 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:48.087659 (+ 29us) client_negotiation.cc:769] Sending connection context
0809 02:16:48.087756 (+ 97us) client_negotiation.cc:240] Negotiation successful
0809 02:16:48.087832 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":789,"mutex_wait_us":88,"thread_start_us":531,"threads_started":1}
I20260809 02:16:48.088346 31984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:48.081527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.129:44729 (local address 127.0.0.1:49454)
0809 02:16:48.082404 (+ 877us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:48.082452 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:48.082596 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:48.083251 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:48.083259 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:48.083286 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:16:48.083640 (+ 354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:48.083658 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:48.085070 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:48.085072 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:48.086470 (+ 1398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:48.086482 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:48.087476 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:48.087483 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:48.087615 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:48.087637 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 02:16:48.087705 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 02:16:48.087757 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":711,"mutex_wait_us":271,"thread_start_us":81,"threads_started":1}
I20260809 02:16:48.091102 31870 tablet_service.cc:1467] Processing CreateTablet for tablet 62d5b0df0994451fa1f33da58131f8a3 (DEFAULT_TABLE table=test-table [id=5d1aa230cf29436089670e8d69e3901d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:16:48.091279 31452 tablet_service.cc:1467] Processing CreateTablet for tablet 62d5b0df0994451fa1f33da58131f8a3 (DEFAULT_TABLE table=test-table [id=5d1aa230cf29436089670e8d69e3901d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:16:48.091279 31665 tablet_service.cc:1467] Processing CreateTablet for tablet 62d5b0df0994451fa1f33da58131f8a3 (DEFAULT_TABLE table=test-table [id=5d1aa230cf29436089670e8d69e3901d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:16:48.092227 31870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62d5b0df0994451fa1f33da58131f8a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:16:48.092278 31665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62d5b0df0994451fa1f33da58131f8a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:16:48.092278 31452 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62d5b0df0994451fa1f33da58131f8a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:16:48.098559 31992 tablet_bootstrap.cc:492] T 62d5b0df0994451fa1f33da58131f8a3 P 285ebddb9b174e1cbfa3a328933a8cf5: Bootstrap starting.
I20260809 02:16:48.099706 31993 tablet_bootstrap.cc:492] T 62d5b0df0994451fa1f33da58131f8a3 P d872f435a0174e0a8995d2305aa18a8f: Bootstrap starting.
I20260809 02:16:48.101011 31994 tablet_bootstrap.cc:492] T 62d5b0df0994451fa1f33da58131f8a3 P 3ca04362b50c4480b5164a01d12f78f9: Bootstrap starting.
I20260809 02:16:48.101131 31992 tablet_bootstrap.cc:654] T 62d5b0df0994451fa1f33da58131f8a3 P 285ebddb9b174e1cbfa3a328933a8cf5: Neither blocks nor log segments found. Creating new log.
I20260809 02:16:48.101639 31993 tablet_bootstrap.cc:654] T 62d5b0df0994451fa1f33da58131f8a3 P d872f435a0174e0a8995d2305aa18a8f: Neither blocks nor log segments found. Creating new log.
I20260809 02:16:48.102312 31992 log.cc:824] T 62d5b0df0994451fa1f33da58131f8a3 P 285ebddb9b174e1cbfa3a328933a8cf5: Log is configured to *not* fsync() on all Append() calls
I20260809 02:16:48.102542 31993 log.cc:824] T 62d5b0df0994451fa1f33da58131f8a3 P d872f435a0174e0a8995d2305aa18a8f: Log is configured to *not* fsync() on all Append() calls
I20260809 02:16:48.103557 31994 tablet_bootstrap.cc:654] T 62d5b0df0994451fa1f33da58131f8a3 P 3ca04362b50c4480b5164a01d12f78f9: Neither blocks nor log segments found. Creating new log.
I20260809 02:16:48.104337 31994 log.cc:824] T 62d5b0df0994451fa1f33da58131f8a3 P 3ca04362b50c4480b5164a01d12f78f9: Log is configured to *not* fsync() on all Append() calls
I20260809 02:16:48.104517 31992 tablet_bootstrap.cc:492] T 62d5b0df0994451fa1f33da58131f8a3 P 285ebddb9b174e1cbfa3a328933a8cf5: No bootstrap required, opened a new log
I20260809 02:16:48.104573 31993 tablet_bootstrap.cc:492] T 62d5b0df0994451fa1f33da58131f8a3 P d872f435a0174e0a8995d2305aa18a8f: No bootstrap required, opened a new log
I20260809 02:16:48.104750 31992 ts_tablet_manager.cc:1397] T 62d5b0df0994451fa1f33da58131f8a3 P 285ebddb9b174e1cbfa3a328933a8cf5: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260809 02:16:48.104784 31993 ts_tablet_manager.cc:1397] T 62d5b0df0994451fa1f33da58131f8a3 P d872f435a0174e0a8995d2305aa18a8f: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260809 02:16:48.106376 31994 tablet_bootstrap.cc:492] T 62d5b0df0994451fa1f33da58131f8a3 P 3ca04362b50c4480b5164a01d12f78f9: No bootstrap required, opened a new log
I20260809 02:16:48.106571 31994 ts_tablet_manager.cc:1397] T 62d5b0df0994451fa1f33da58131f8a3 P 3ca04362b50c4480b5164a01d12f78f9: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 02:16:48.107743 31992 raft_consensus.cc:348] T 62d5b0df0994451fa1f33da58131f8a3 P 285ebddb9b174e1cbfa3a328933a8cf5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ca04362b50c4480b5164a01d12f78f9" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 33849 } } peers { permanent_uuid: "d872f435a0174e0a8995d2305aa18a8f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 44729 } } peers { permanent_uuid: "285ebddb9b174e1cbfa3a328933a8cf5" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 40923 } }
I20260809 02:16:48.107975 31992 raft_consensus.cc:374] T 62d5b0df0994451fa1f33da58131f8a3 P 285ebddb9b174e1cbfa3a328933a8cf5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:16:48.108029 31992 raft_consensus.cc:729] T 62d5b0df0994451fa1f33da58131f8a3 P 285ebddb9b174e1cbfa3a328933a8cf5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 285ebddb9b174e1cbfa3a328933a8cf5, State: Initialized, Role: FOLLOWER
I20260809 02:16:48.108476 31992 consensus_queue.cc:260] T 62d5b0df0994451fa1f33da58131f8a3 P 285ebddb9b174e1cbfa3a328933a8cf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ca04362b50c4480b5164a01d12f78f9" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 33849 } } peers { permanent_uuid: "d872f435a0174e0a8995d2305aa18a8f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 44729 } } peers { permanent_uuid: "285ebddb9b174e1cbfa3a328933a8cf5" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 40923 } }
I20260809 02:16:48.109166 31970 heartbeater.cc:503] Master 127.15.75.190:40319 was elected leader, sending a full tablet report...
I20260809 02:16:48.109318 31993 raft_consensus.cc:348] T 62d5b0df0994451fa1f33da58131f8a3 P d872f435a0174e0a8995d2305aa18a8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ca04362b50c4480b5164a01d12f78f9" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 33849 } } peers { permanent_uuid: "d872f435a0174e0a8995d2305aa18a8f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 44729 } } peers { permanent_uuid: "285ebddb9b174e1cbfa3a328933a8cf5" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 40923 } }
I20260809 02:16:48.109557 31993 raft_consensus.cc:374] T 62d5b0df0994451fa1f33da58131f8a3 P d872f435a0174e0a8995d2305aa18a8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:16:48.109592 31992 ts_tablet_manager.cc:1428] T 62d5b0df0994451fa1f33da58131f8a3 P 285ebddb9b174e1cbfa3a328933a8cf5: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260809 02:16:48.109613 31993 raft_consensus.cc:729] T 62d5b0df0994451fa1f33da58131f8a3 P d872f435a0174e0a8995d2305aa18a8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d872f435a0174e0a8995d2305aa18a8f, State: Initialized, Role: FOLLOWER
I20260809 02:16:48.110065 31993 consensus_queue.cc:260] T 62d5b0df0994451fa1f33da58131f8a3 P d872f435a0174e0a8995d2305aa18a8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ca04362b50c4480b5164a01d12f78f9" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 33849 } } peers { permanent_uuid: "d872f435a0174e0a8995d2305aa18a8f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 44729 } } peers { permanent_uuid: "285ebddb9b174e1cbfa3a328933a8cf5" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 40923 } }
I20260809 02:16:48.110430 31994 raft_consensus.cc:348] T 62d5b0df0994451fa1f33da58131f8a3 P 3ca04362b50c4480b5164a01d12f78f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ca04362b50c4480b5164a01d12f78f9" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 33849 } } peers { permanent_uuid: "d872f435a0174e0a8995d2305aa18a8f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 44729 } } peers { permanent_uuid: "285ebddb9b174e1cbfa3a328933a8cf5" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 40923 } }
I20260809 02:16:48.110653 31994 raft_consensus.cc:374] T 62d5b0df0994451fa1f33da58131f8a3 P 3ca04362b50c4480b5164a01d12f78f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:16:48.110698 31994 raft_consensus.cc:729] T 62d5b0df0994451fa1f33da58131f8a3 P 3ca04362b50c4480b5164a01d12f78f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ca04362b50c4480b5164a01d12f78f9, State: Initialized, Role: FOLLOWER
I20260809 02:16:48.111063 31993 ts_tablet_manager.cc:1428] T 62d5b0df0994451fa1f33da58131f8a3 P d872f435a0174e0a8995d2305aa18a8f: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 02:16:48.111131 31994 consensus_queue.cc:260] T 62d5b0df0994451fa1f33da58131f8a3 P 3ca04362b50c4480b5164a01d12f78f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ca04362b50c4480b5164a01d12f78f9" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 33849 } } peers { permanent_uuid: "d872f435a0174e0a8995d2305aa18a8f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 44729 } } peers { permanent_uuid: "285ebddb9b174e1cbfa3a328933a8cf5" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 40923 } }
I20260809 02:16:48.111480 31555 heartbeater.cc:503] Master 127.15.75.190:40319 was elected leader, sending a full tablet report...
I20260809 02:16:48.112149 31766 heartbeater.cc:503] Master 127.15.75.190:40319 was elected leader, sending a full tablet report...
I20260809 02:16:48.112255 31994 ts_tablet_manager.cc:1428] T 62d5b0df0994451fa1f33da58131f8a3 P 3ca04362b50c4480b5164a01d12f78f9: Time spent starting tablet: real 0.006s user 0.001s sys 0.005s
I20260809 02:16:48.173804 32004 raft_consensus.cc:482] T 62d5b0df0994451fa1f33da58131f8a3 P 285ebddb9b174e1cbfa3a328933a8cf5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:16:48.174010 32004 raft_consensus.cc:504] T 62d5b0df0994451fa1f33da58131f8a3 P 285ebddb9b174e1cbfa3a328933a8cf5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ca04362b50c4480b5164a01d12f78f9" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 33849 } } peers { permanent_uuid: "d872f435a0174e0a8995d2305aa18a8f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 44729 } } peers { permanent_uuid: "285ebddb9b174e1cbfa3a328933a8cf5" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 40923 } }
I20260809 02:16:48.175100 32004 leader_election.cc:288] T 62d5b0df0994451fa1f33da58131f8a3 P 285ebddb9b174e1cbfa3a328933a8cf5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ca04362b50c4480b5164a01d12f78f9 (127.15.75.130:33849), d872f435a0174e0a8995d2305aa18a8f (127.15.75.129:44729)
I20260809 02:16:48.179513 31989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:48.175546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.131:59519 (local address 127.15.75.129:44729)
0809 02:16:48.175696 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:48.175703 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:48.175719 (+ 16us) server_negotiation.cc:407] Connection header received
0809 02:16:48.175775 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:48.175777 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:48.175823 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:48.175907 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:16:48.176297 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:48.176934 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:48.178340 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:48.178930 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:48.179072 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:48.179128 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:48.179314 (+ 186us) server_negotiation.cc:299] Negotiation successful
0809 02:16:48.179385 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 02:16:48.179543 31973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:48.175353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.129:44729 (local address 127.15.75.131:59519)
0809 02:16:48.175496 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:48.175513 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:48.175616 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:48.175935 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:48.175938 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:48.175974 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:16:48.176156 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:48.176163 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:48.177041 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:48.177045 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:48.178226 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:48.178237 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:48.179060 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:48.179063 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:48.179176 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:48.179198 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 02:16:48.179297 (+ 99us) client_negotiation.cc:240] Negotiation successful
0809 02:16:48.179376 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53,"mutex_wait_us":6,"spinlock_wait_cycles":8448}
I20260809 02:16:48.179579 31987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:48.175588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.131:46111 (local address 127.15.75.130:33849)
0809 02:16:48.175717 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:48.175724 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:48.175996 (+ 272us) server_negotiation.cc:407] Connection header received
0809 02:16:48.176124 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:48.176128 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:48.176179 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:48.176258 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:16:48.176538 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:48.177133 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:48.178485 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:48.179007 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:48.179141 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:48.179194 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:48.179379 (+ 185us) server_negotiation.cc:299] Negotiation successful
0809 02:16:48.179457 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 02:16:48.180047 32008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:48.175353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.130:33849 (local address 127.15.75.131:46111)
0809 02:16:48.175887 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:48.175912 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:48.176007 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:48.176289 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:48.176292 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:48.176309 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:16:48.176424 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:48.176431 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:48.177248 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:48.177251 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:48.178322 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:48.178335 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:48.179137 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:48.179140 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:48.179248 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:48.179275 (+ 27us) client_negotiation.cc:769] Sending connection context
0809 02:16:48.179363 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 02:16:48.179431 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"mutex_wait_us":150,"thread_start_us":126,"threads_started":1}
I20260809 02:16:48.180241 31482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62d5b0df0994451fa1f33da58131f8a3" candidate_uuid: "285ebddb9b174e1cbfa3a328933a8cf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d872f435a0174e0a8995d2305aa18a8f" is_pre_election: true
I20260809 02:16:48.180594 31482 raft_consensus.cc:2454] T 62d5b0df0994451fa1f33da58131f8a3 P d872f435a0174e0a8995d2305aa18a8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 285ebddb9b174e1cbfa3a328933a8cf5 in term 0.
I20260809 02:16:48.180670 31697 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62d5b0df0994451fa1f33da58131f8a3" candidate_uuid: "285ebddb9b174e1cbfa3a328933a8cf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ca04362b50c4480b5164a01d12f78f9" is_pre_election: true
I20260809 02:16:48.180985 31697 raft_consensus.cc:2454] T 62d5b0df0994451fa1f33da58131f8a3 P 3ca04362b50c4480b5164a01d12f78f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 285ebddb9b174e1cbfa3a328933a8cf5 in term 0.
I20260809 02:16:48.181149 31805 leader_election.cc:302] T 62d5b0df0994451fa1f33da58131f8a3 P 285ebddb9b174e1cbfa3a328933a8cf5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 285ebddb9b174e1cbfa3a328933a8cf5, d872f435a0174e0a8995d2305aa18a8f; no voters:
I20260809 02:16:48.181463 32004 raft_consensus.cc:2781] T 62d5b0df0994451fa1f33da58131f8a3 P 285ebddb9b174e1cbfa3a328933a8cf5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:16:48.181560 32004 raft_consensus.cc:482] T 62d5b0df0994451fa1f33da58131f8a3 P 285ebddb9b174e1cbfa3a328933a8cf5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:16:48.181623 32004 raft_consensus.cc:3037] T 62d5b0df0994451fa1f33da58131f8a3 P 285ebddb9b174e1cbfa3a328933a8cf5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:16:48.182649 32004 raft_consensus.cc:504] T 62d5b0df0994451fa1f33da58131f8a3 P 285ebddb9b174e1cbfa3a328933a8cf5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ca04362b50c4480b5164a01d12f78f9" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 33849 } } peers { permanent_uuid: "d872f435a0174e0a8995d2305aa18a8f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 44729 } } peers { permanent_uuid: "285ebddb9b174e1cbfa3a328933a8cf5" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 40923 } }
I20260809 02:16:48.182998 32004 leader_election.cc:288] T 62d5b0df0994451fa1f33da58131f8a3 P 285ebddb9b174e1cbfa3a328933a8cf5 [CANDIDATE]: Term 1 election: Requested vote from peers 3ca04362b50c4480b5164a01d12f78f9 (127.15.75.130:33849), d872f435a0174e0a8995d2305aa18a8f (127.15.75.129:44729)
I20260809 02:16:48.183355 31697 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62d5b0df0994451fa1f33da58131f8a3" candidate_uuid: "285ebddb9b174e1cbfa3a328933a8cf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ca04362b50c4480b5164a01d12f78f9"
I20260809 02:16:48.183362 31482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62d5b0df0994451fa1f33da58131f8a3" candidate_uuid: "285ebddb9b174e1cbfa3a328933a8cf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d872f435a0174e0a8995d2305aa18a8f"
I20260809 02:16:48.183480 31697 raft_consensus.cc:3037] T 62d5b0df0994451fa1f33da58131f8a3 P 3ca04362b50c4480b5164a01d12f78f9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:16:48.183480 31482 raft_consensus.cc:3037] T 62d5b0df0994451fa1f33da58131f8a3 P d872f435a0174e0a8995d2305aa18a8f [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:16:48.184779 31697 raft_consensus.cc:2454] T 62d5b0df0994451fa1f33da58131f8a3 P 3ca04362b50c4480b5164a01d12f78f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 285ebddb9b174e1cbfa3a328933a8cf5 in term 1.
I20260809 02:16:48.184779 31482 raft_consensus.cc:2454] T 62d5b0df0994451fa1f33da58131f8a3 P d872f435a0174e0a8995d2305aa18a8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 285ebddb9b174e1cbfa3a328933a8cf5 in term 1.
I20260809 02:16:48.185067 31801 leader_election.cc:302] T 62d5b0df0994451fa1f33da58131f8a3 P 285ebddb9b174e1cbfa3a328933a8cf5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 285ebddb9b174e1cbfa3a328933a8cf5, 3ca04362b50c4480b5164a01d12f78f9, d872f435a0174e0a8995d2305aa18a8f; no voters:
I20260809 02:16:48.185245 32004 raft_consensus.cc:2781] T 62d5b0df0994451fa1f33da58131f8a3 P 285ebddb9b174e1cbfa3a328933a8cf5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:16:48.185490 32004 raft_consensus.cc:686] T 62d5b0df0994451fa1f33da58131f8a3 P 285ebddb9b174e1cbfa3a328933a8cf5 [term 1 LEADER]: Becoming Leader. State: Replica: 285ebddb9b174e1cbfa3a328933a8cf5, State: Running, Role: LEADER
I20260809 02:16:48.185806 32004 consensus_queue.cc:237] T 62d5b0df0994451fa1f33da58131f8a3 P 285ebddb9b174e1cbfa3a328933a8cf5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ca04362b50c4480b5164a01d12f78f9" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 33849 } } peers { permanent_uuid: "d872f435a0174e0a8995d2305aa18a8f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 44729 } } peers { permanent_uuid: "285ebddb9b174e1cbfa3a328933a8cf5" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 40923 } }
I20260809 02:16:48.189080 31259 catalog_manager.cc:5495] T 62d5b0df0994451fa1f33da58131f8a3 P 285ebddb9b174e1cbfa3a328933a8cf5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 285ebddb9b174e1cbfa3a328933a8cf5 (127.15.75.131). New cstate: current_term: 1 leader_uuid: "285ebddb9b174e1cbfa3a328933a8cf5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ca04362b50c4480b5164a01d12f78f9" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 33849 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d872f435a0174e0a8995d2305aa18a8f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 44729 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "285ebddb9b174e1cbfa3a328933a8cf5" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 40923 } health_report { overall_health: HEALTHY } } }
I20260809 02:16:48.209532 31990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:48.206317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34346 (local address 127.15.75.131:40923)
0809 02:16:48.206439 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:48.206442 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:48.206454 (+ 12us) server_negotiation.cc:407] Connection header received
0809 02:16:48.206586 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:48.206588 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:48.206639 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:48.206723 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:16:48.206957 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:48.207437 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:48.208235 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:48.208442 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:48.208514 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:48.208831 (+ 317us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:16:48.209227 (+ 396us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:16:48.209315 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:48.209379 (+ 64us) server_negotiation.cc:299] Negotiation successful
0809 02:16:48.209435 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 02:16:48.215162 31482 raft_consensus.cc:1261] T 62d5b0df0994451fa1f33da58131f8a3 P d872f435a0174e0a8995d2305aa18a8f [term 1 FOLLOWER]: Refusing update from remote peer 285ebddb9b174e1cbfa3a328933a8cf5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:16:48.215191 31697 raft_consensus.cc:1261] T 62d5b0df0994451fa1f33da58131f8a3 P 3ca04362b50c4480b5164a01d12f78f9 [term 1 FOLLOWER]: Refusing update from remote peer 285ebddb9b174e1cbfa3a328933a8cf5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:16:48.215873 32009 consensus_queue.cc:1035] T 62d5b0df0994451fa1f33da58131f8a3 P 285ebddb9b174e1cbfa3a328933a8cf5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ca04362b50c4480b5164a01d12f78f9" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 33849 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:16:48.216032 32004 consensus_queue.cc:1035] T 62d5b0df0994451fa1f33da58131f8a3 P 285ebddb9b174e1cbfa3a328933a8cf5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d872f435a0174e0a8995d2305aa18a8f" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 44729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:16:48.225010 32015 mvcc.cc:205] Tried to move back new op lower bound from 7316446446442504192 to 7316446446331691008. Current Snapshot: MvccSnapshot[applied={T|T < 7316446446442504192}]
I20260809 02:16:48.223457 32014 mvcc.cc:205] Tried to move back new op lower bound from 7316446446442504192 to 7316446446331691008. Current Snapshot: MvccSnapshot[applied={T|T < 7316446446442504192}]
I20260809 02:16:48.233991 31262 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60538:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:16:48.234127 31262 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60538:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:16:48.236490 31262 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 74715eccb6c343149c3ee8d054201318: Sending DeleteTablet for 3 replicas of tablet 62d5b0df0994451fa1f33da58131f8a3
I20260809 02:16:48.237334 31665 tablet_service.cc:1514] Processing DeleteTablet for tablet 62d5b0df0994451fa1f33da58131f8a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:16:48 UTC) from {username='kudu', principal='kudu/127.15.75.190@KRBTEST.COM'} at 127.0.0.1:56830
I20260809 02:16:48.237847 32025 tablet_replica.cc:330] T 62d5b0df0994451fa1f33da58131f8a3 P 3ca04362b50c4480b5164a01d12f78f9: stopping tablet replica
I20260809 02:16:48.237876 31452 tablet_service.cc:1514] Processing DeleteTablet for tablet 62d5b0df0994451fa1f33da58131f8a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:16:48 UTC) from {username='kudu', principal='kudu/127.15.75.190@KRBTEST.COM'} at 127.0.0.1:49454
I20260809 02:16:48.238036 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 31355
I20260809 02:16:48.238720 31870 tablet_service.cc:1514] Processing DeleteTablet for tablet 62d5b0df0994451fa1f33da58131f8a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:16:48 UTC) from {username='kudu', principal='kudu/127.15.75.190@KRBTEST.COM'} at 127.0.0.1:34334
I20260809 02:16:48.242487 32025 raft_consensus.cc:2229] T 62d5b0df0994451fa1f33da58131f8a3 P 3ca04362b50c4480b5164a01d12f78f9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:16:48.242846 32025 raft_consensus.cc:2258] T 62d5b0df0994451fa1f33da58131f8a3 P 3ca04362b50c4480b5164a01d12f78f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:16:48.244450 32025 ts_tablet_manager.cc:1905] T 62d5b0df0994451fa1f33da58131f8a3 P 3ca04362b50c4480b5164a01d12f78f9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:16:48.245617 32026 tablet_replica.cc:330] T 62d5b0df0994451fa1f33da58131f8a3 P 285ebddb9b174e1cbfa3a328933a8cf5: stopping tablet replica
I20260809 02:16:48.246033 32026 raft_consensus.cc:2229] T 62d5b0df0994451fa1f33da58131f8a3 P 285ebddb9b174e1cbfa3a328933a8cf5 [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:16:48.246534 32026 raft_consensus.cc:2258] T 62d5b0df0994451fa1f33da58131f8a3 P 285ebddb9b174e1cbfa3a328933a8cf5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:16:48.247712 32025 ts_tablet_manager.cc:1918] T 62d5b0df0994451fa1f33da58131f8a3 P 3ca04362b50c4480b5164a01d12f78f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:16:48.247869 32025 log.cc:1196] T 62d5b0df0994451fa1f33da58131f8a3 P 3ca04362b50c4480b5164a01d12f78f9: Deleting WAL directory at /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/minicluster-data/ts-1/wal/wals/62d5b0df0994451fa1f33da58131f8a3
I20260809 02:16:48.248340 32025 ts_tablet_manager.cc:1939] T 62d5b0df0994451fa1f33da58131f8a3 P 3ca04362b50c4480b5164a01d12f78f9: Deleting consensus metadata
I20260809 02:16:48.249308 32026 ts_tablet_manager.cc:1905] T 62d5b0df0994451fa1f33da58131f8a3 P 285ebddb9b174e1cbfa3a328933a8cf5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:16:48.249791 31233 catalog_manager.cc:4841] TS 3ca04362b50c4480b5164a01d12f78f9 (127.15.75.130:33849): tablet 62d5b0df0994451fa1f33da58131f8a3 (table test-table [id=5d1aa230cf29436089670e8d69e3901d]) successfully deleted
I20260809 02:16:48.253180 32026 ts_tablet_manager.cc:1918] T 62d5b0df0994451fa1f33da58131f8a3 P 285ebddb9b174e1cbfa3a328933a8cf5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:16:48.253309 32026 log.cc:1196] T 62d5b0df0994451fa1f33da58131f8a3 P 285ebddb9b174e1cbfa3a328933a8cf5: Deleting WAL directory at /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241741788589-15662-0/minicluster-data/ts-2/wal/wals/62d5b0df0994451fa1f33da58131f8a3
I20260809 02:16:48.253700 32026 ts_tablet_manager.cc:1939] T 62d5b0df0994451fa1f33da58131f8a3 P 285ebddb9b174e1cbfa3a328933a8cf5: Deleting consensus metadata
I20260809 02:16:48.266052 31239 catalog_manager.cc:4841] TS 285ebddb9b174e1cbfa3a328933a8cf5 (127.15.75.131:40923): tablet 62d5b0df0994451fa1f33da58131f8a3 (table test-table [id=5d1aa230cf29436089670e8d69e3901d]) successfully deleted
W20260809 02:16:48.272442 31240 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.75.129:44729 (error 108)
I20260809 02:16:48.273429 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 31561
I20260809 02:16:48.273670 31984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:48.273322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.129:44729 (local address 127.0.0.1:49464)
0809 02:16:48.273490 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:48.273569 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.129:44729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260809 02:16:48.281009 31240 catalog_manager.cc:4584] TS d872f435a0174e0a8995d2305aa18a8f (127.15.75.129:44729): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 62d5b0df0994451fa1f33da58131f8a3: Network error: Client connection negotiation failed: client connection to 127.15.75.129:44729: connect: Connection refused (error 111)
I20260809 02:16:48.288339 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 31775
I20260809 02:16:48.298056 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 31206
2026-08-09T02:16:48Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2590 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:16:48 dist-test-slave-s11t krb5kdc[32030](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:16:48 dist-test-slave-s11t krb5kdc[32030](info): set up 2 sockets
Aug 09 02:16:48 dist-test-slave-s11t krb5kdc[32030](info): commencing operation
krb5kdc: starting...
W20260809 02:16:49.541077 15662 mini_kdc.cc:121] Time spent starting KDC: real 1.215s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Aug 09 02:16:49 dist-test-slave-s11t krb5kdc[32030](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241809, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:16:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:16:49Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.15.75.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.15.75.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.15.75.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/krb5kdc/rangeradmin_127.15.75.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.15.75.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/krb5kdc/rangeradmin_127.15.75.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.15.75.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.15.75.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.15.75.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/krb5kdc/rangerlookup_127.15.75.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.15.75.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/krb5kdc/rangerlookup_127.15.75.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.15.75.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.15.75.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.15.75.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/krb5kdc/HTTP_127.15.75.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.15.75.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/krb5kdc/HTTP_127.15.75.148@KRBTEST.COM.keytab.
I20260809 02:16:49.713570 15662 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-tasktcWNU7/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/postgres -l logfile start
2026-08-09 02:16:52.098 UTC [32080] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 02:16:52.098 UTC [32080] LOG: listening on IPv4 address "127.15.75.148", port 53663
2026-08-09 02:16:52.101 UTC [32080] LOG: listening on Unix socket "/tmp/.s.PGSQL.53663"
2026-08-09 02:16:52.139 UTC [32083] LOG: database system was shut down at 2026-08-09 02:16:50 UTC
2026-08-09 02:16:52.143 UTC [32080] LOG: database system is ready to accept connections
I20260809 02:16:53.332139 15662 mini_postgres.cc:96] Postgres bound to 53663
2026-08-09 02:16:53.339 UTC [32095] FATAL: database "slave" does not exist
127.15.75.148:53663 - accepting connections
I20260809 02:16:53.340497 15662 mini_ranger.cc:161] Starting Ranger...
I20260809 02:16:53.356168 15662 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 02:16:53.786880 15662 mini_ranger.cc:87] Created ranger Postgres database
I20260809 02:16:53.786985 15662 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/ranger-admin
2026-08-09 02:16:54,119 [I] DB FLAVOR :POSTGRES
2026-08-09 02:16:54,119 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 02:16:54,119 [I] Checking connection..
2026-08-09 02:16:54,120 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktcWNU7/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktcWNU7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.75.148:53663/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 02:16:54,392 [I] Checking connection passed.
2026-08-09 02:16:54,392 [I] --------- Verifying version history table ---------
2026-08-09 02:16:54,393 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktcWNU7/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktcWNU7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.75.148:53663/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 02:16:54,632 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 02:16:54,632 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktcWNU7/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktcWNU7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.75.148:53663/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 02:16:54,880 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 02:16:54,880 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 02:16:54,880 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktcWNU7/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktcWNU7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.75.148:53663/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasktcWNU7/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 02:16:55.108 UTC [32199] WARNING: there is no transaction in progress
2026-08-09 02:16:55,117 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 02:16:55,117 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktcWNU7/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktcWNU7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.75.148:53663/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 02:16:55,362 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 02:16:55,362 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 02:16:55,362 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktcWNU7/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktcWNU7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.75.148:53663/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 02:16:55,611 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktcWNU7/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktcWNU7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.75.148:53663/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 02:16:55,868 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktcWNU7/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktcWNU7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.75.148:53663/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-s11t.c.gcp-upstream.internal','N') ;"
2026-08-09 02:16:56,114 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktcWNU7/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktcWNU7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.75.148:53663/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 02:16:56,364 [I] Table x_portal_user does not exist in database ranger
2026-08-09 02:16:56,364 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktcWNU7/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktcWNU7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.75.148:53663/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 02:16:56,630 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 02:16:56,630 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktcWNU7/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktcWNU7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.75.148:53663/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 02:16:56,889 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktcWNU7/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktcWNU7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.75.148:53663/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 02:16:57,122 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 02:16:57,123 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktcWNU7/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktcWNU7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.75.148:53663/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasktcWNU7/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 02:16:58.067 UTC [32363] WARNING: there is no transaction in progress
2026-08-09 02:16:58.093 UTC [32363] WARNING: there is no transaction in progress
2026-08-09 02:16:58.109 UTC [32363] WARNING: there is no transaction in progress
2026-08-09 02:16:58.123 UTC [32363] WARNING: there is no transaction in progress
2026-08-09 02:16:58.138 UTC [32363] WARNING: there is no transaction in progress
2026-08-09 02:16:58.154 UTC [32363] WARNING: there is no transaction in progress
2026-08-09 02:16:58.209 UTC [32363] WARNING: there is no transaction in progress
2026-08-09 02:16:58.215 UTC [32363] WARNING: there is no transaction in progress
2026-08-09 02:16:58.229 UTC [32363] WARNING: there is no transaction in progress
2026-08-09 02:16:58.239 UTC [32363] WARNING: there is no transaction in progress
2026-08-09 02:16:58.249 UTC [32363] WARNING: there is no transaction in progress
2026-08-09 02:16:58.262 UTC [32363] WARNING: there is no transaction in progress
2026-08-09 02:16:58.271 UTC [32363] WARNING: there is no transaction in progress
2026-08-09 02:16:58.277 UTC [32363] WARNING: there is no transaction in progress
2026-08-09 02:16:58.284 UTC [32363] WARNING: there is no transaction in progress
2026-08-09 02:16:58.290 UTC [32363] WARNING: there is no transaction in progress
2026-08-09 02:16:58,893 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 02:16:58,894 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktcWNU7/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktcWNU7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.75.148:53663/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 02:16:59,156 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 02:16:59,157 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktcWNU7/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktcWNU7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.75.148:53663/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 02:16:59,406 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 02:16:59,407 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktcWNU7/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktcWNU7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.75.148:53663/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 02:16:59,650 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 02:16:59,650 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktcWNU7/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktcWNU7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.75.148:53663/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 02:16:59,892 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktcWNU7/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktcWNU7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.75.148:53663/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 02:17:00,141 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktcWNU7/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktcWNU7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.75.148:53663/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-s11t.c.gcp-upstream.internal';"
2026-08-09 02:17:00,392 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 02:17:00,393 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktcWNU7/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktcWNU7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.75.148:53663/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 02:17:00,667 [I] DB_PATCHES have already been applied
I20260809 02:17:00.673436 15662 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/ranger-admin:/tmp/dist-test-tasktcWNU7/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-tasktcWNU7/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-tasktcWNU7/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktcWNU7/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 02:17:00.673502 15662 mini_ranger.cc:193] Using host: 127.15.75.148
I20260809 02:17:00.677937 15662 mini_ranger.cc:227] Ranger admin URL: http://127.15.75.148:59969
Aug 09, 2026 2:17:00 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 2:17:00 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 2:17:01 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 2:17:01 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 2:17:01 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 2:17:01 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 2:17:01 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 02:17:02.065851 15662 mini_ranger.cc:160] Time spent starting Ranger: real 8.725s user 0.002s sys 0.010s
Aug 09, 2026 2:17:22 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 2:17:22 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 09, 2026 2:17:22 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 2:17:22 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 2:17:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 2:17:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 2:17:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 2:17:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 2:17:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 2:17:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 2:17:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 2:17:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 2:17:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 2:17:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 2:17:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 2:17:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 2:17:22 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 2:17:23 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 02:17:24.212227 15662 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.15.75.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.15.75.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.15.75.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/krb5kdc/rangerkms_127.15.75.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.15.75.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/krb5kdc/rangerkms_127.15.75.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.15.75.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.15.75.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.15.75.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/krb5kdc/HTTP_127.15.75.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.15.75.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/krb5kdc/HTTP_127.15.75.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 02:17:24 dist-test-slave-s11t krb5kdc[32030](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241844, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 02:17:24.307071 15662 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 02:17:24.317839 15662 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 02:17:25.105823 15662 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 02:17:25.105929 15662 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/ranger-kms
I20260809 02:17:25.106058 15662 mini_ranger_kms.cc:227] Using postgres at 127.15.75.148:53663
2026-08-09 02:17:25,668 [I] DB FLAVOR :POSTGRES
2026-08-09 02:17:25,669 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 02:17:25,669 [I] Checking connection
2026-08-09 02:17:25,669 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktcWNU7/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.75.148:53663/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 02:17:25,924 [I] connection success
2026-08-09 02:17:25,924 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 02:17:25,924 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 02:17:25,924 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktcWNU7/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.75.148:53663/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 02:17:26,164 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 02:17:26,164 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 02:17:26,164 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 02:17:26,164 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktcWNU7/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.75.148:53663/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 02:17:26,454 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 02:17:26.949183 15662 mini_ranger_kms.cc:310] Created kms service
I20260809 02:17:27.179673 15662 mini_ranger_kms.cc:326] Created kudu user
I20260809 02:17:27.225596 15662 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 02:17:27 dist-test-slave-s11t krb5kdc[32030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241844, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.15.75.148@KRBTEST.COM
I20260809 02:17:27.632848 15662 mini_ranger_kms.cc:384] Added ranger policy
I20260809 02:17:27.633008 15662 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/ranger-kms:/tmp/dist-test-tasktcWNU7/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktcWNU7/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-tasktcWNU7/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 02:17:27.633029 15662 mini_ranger_kms.cc:240] Using host: 127.15.75.148
I20260809 02:17:27.638837 15662 mini_ranger_kms.cc:276] Ranger KMS PID: 32650
I20260809 02:17:27.638985 15662 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.15.75.148:39187
Aug 09, 2026 2:17:28 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 2:17:28 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 2:17:28 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 2:17:28 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 2:17:29 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 2:17:29 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 09 02:17:30 dist-test-slave-s11t krb5kdc[32030](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786241850, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.75.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:17:30.303895 15662 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.997s user 0.024s sys 0.234s
I20260809 02:17:30.304034 15662 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 02:17:30.304140 15662 mini_ranger_kms.cc:401] 127.15.75.148:39187/kms/v1/keys
Aug 09 02:17:30 dist-test-slave-s11t krb5kdc[32030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241850, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.75.148@KRBTEST.COM for HTTP/127.15.75.148@KRBTEST.COM
Aug 09 02:17:30 dist-test-slave-s11t krb5kdc[32030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241850, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.75.148@KRBTEST.COM for HTTP/127.15.75.148@KRBTEST.COM
Aug 09 02:17:33 dist-test-slave-s11t krb5kdc[32030](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241853, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.15.75.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab.
Entry for principal kudu/127.15.75.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab.
I20260809 02:17:33.221122 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:36003
--webserver_interface=127.15.75.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab
--principal=kudu/127.15.75.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:40839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.75.190:36003
--ranger_config_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.75.148:39187/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:17:33.349329 32725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:17:33.349581 32725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:17:33.349619 32725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:17:33.353441 32725 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:17:33.353487 32725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:17:33.353505 32725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:17:33.353521 32725 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:17:33.353536 32725 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:17:33.358810 32725 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:40839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/master-0/wal
--ranger_kms_url=127.15.75.148:39187/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.15.75.190:36003
--ranger_config_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab
--principal=kudu/127.15.75.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:36003
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.75.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32725
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:17:33.359700 32725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:17:33.360615 32725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:17:33.369481 32736 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:17:33.369498 32733 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:17:33.369491 32734 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:17:33.369544 32725 server_base.cc:1034] running on GCE node
I20260809 02:17:33.370352 32725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:17:33.371392 32725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:17:33.372553 32725 hybrid_clock.cc:648] HybridClock initialized: now 1786241853372533 us; error 30 us; skew 500 ppm
Aug 09 02:17:33 dist-test-slave-s11t krb5kdc[32030](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241853, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:17:33.375341 32725 init.cc:377] Logged in from keytab as kudu/127.15.75.190@KRBTEST.COM (short username kudu)
I20260809 02:17:33.376134 32725 webserver.cc:460] Webserver started at http://127.15.75.190:39705/ using document root <none> and password file <none>
I20260809 02:17:33.376621 32725 fs_manager.cc:359] Metadata directory not provided
I20260809 02:17:33.376672 32725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:17:33.376973 32725 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 02:17:33 dist-test-slave-s11t krb5kdc[32030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241853, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.190@KRBTEST.COM for HTTP/127.15.75.148@KRBTEST.COM
I20260809 02:17:33.429965 32725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/master-0/data/instance:
uuid: "83b05a51148d47e49ea18e04f4037d25"
format_stamp: "Formatted at 2026-08-09 02:17:33 on dist-test-slave-s11t"
server_key: "8f7c5bad8806e67f8070bec557cf0030"
server_key_iv: "54997c3c2e3cb5113e5e25e0342056e6"
server_key_version: "kuduclusterkey@0"
I20260809 02:17:33.430759 32725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/master-0/wal/instance:
uuid: "83b05a51148d47e49ea18e04f4037d25"
format_stamp: "Formatted at 2026-08-09 02:17:33 on dist-test-slave-s11t"
server_key: "8f7c5bad8806e67f8070bec557cf0030"
server_key_iv: "54997c3c2e3cb5113e5e25e0342056e6"
server_key_version: "kuduclusterkey@0"
I20260809 02:17:33.435976 32725 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 02:17:33.452924 32746 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:17:33.454507 32725 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 02:17:33.454643 32725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/master-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/master-0/wal
uuid: "83b05a51148d47e49ea18e04f4037d25"
format_stamp: "Formatted at 2026-08-09 02:17:33 on dist-test-slave-s11t"
server_key: "8f7c5bad8806e67f8070bec557cf0030"
server_key_iv: "54997c3c2e3cb5113e5e25e0342056e6"
server_key_version: "kuduclusterkey@0"
I20260809 02:17:33.454777 32725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:17:33.475427 32725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:17:33.478888 32725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:17:33.479091 32725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:17:33.487637 358 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.190:36003 every 8 connection(s)
I20260809 02:17:33.487648 32725 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.190:36003
I20260809 02:17:33.488945 32725 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
I20260809 02:17:33.490725 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 32725
I20260809 02:17:33.490860 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/master-0/wal/instance
I20260809 02:17:33.491166 15662 external_mini_cluster.cc:1428] Setting key a5567187a22ccc55aa5a94ef7de52a1a
I20260809 02:17:33.492420 359 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:17:33.498714 359 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83b05a51148d47e49ea18e04f4037d25: Bootstrap starting.
I20260809 02:17:33.501446 359 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 83b05a51148d47e49ea18e04f4037d25: Neither blocks nor log segments found. Creating new log.
Aug 09 02:17:33 dist-test-slave-s11t krb5kdc[32030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241853, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:17:33.502293 359 log.cc:824] T 00000000000000000000000000000000 P 83b05a51148d47e49ea18e04f4037d25: Log is configured to *not* fsync() on all Append() calls
I20260809 02:17:33.504947 359 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83b05a51148d47e49ea18e04f4037d25: No bootstrap required, opened a new log
I20260809 02:17:33.507687 366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:33.492550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42012 (local address 127.15.75.190:36003)
0809 02:17:33.493041 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:33.493064 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:33.493110 (+ 46us) server_negotiation.cc:407] Connection header received
0809 02:17:33.493985 (+ 875us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:33.494009 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:33.494430 (+ 421us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:33.494787 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:17:33.495676 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:33.497270 (+ 1594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:33.498446 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:33.499152 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:33.499316 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:33.502348 (+ 3032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:17:33.502355 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:17:33.502373 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:17:33.502412 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:17:33.504842 (+ 2430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:33.505365 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:33.505373 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:33.505380 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:33.505513 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:33.505801 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:33.505806 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:33.505808 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:33.506193 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:17:33.506384 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:33.506718 (+ 334us) server_negotiation.cc:299] Negotiation successful
0809 02:17:33.507009 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":184,"threads_started":1}
I20260809 02:17:33.508898 359 raft_consensus.cc:348] T 00000000000000000000000000000000 P 83b05a51148d47e49ea18e04f4037d25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83b05a51148d47e49ea18e04f4037d25" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 36003 } }
I20260809 02:17:33.509187 359 raft_consensus.cc:374] T 00000000000000000000000000000000 P 83b05a51148d47e49ea18e04f4037d25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:17:33.509284 359 raft_consensus.cc:729] T 00000000000000000000000000000000 P 83b05a51148d47e49ea18e04f4037d25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83b05a51148d47e49ea18e04f4037d25, State: Initialized, Role: FOLLOWER
I20260809 02:17:33.509743 359 consensus_queue.cc:260] T 00000000000000000000000000000000 P 83b05a51148d47e49ea18e04f4037d25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83b05a51148d47e49ea18e04f4037d25" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 36003 } }
I20260809 02:17:33.509979 359 raft_consensus.cc:388] T 00000000000000000000000000000000 P 83b05a51148d47e49ea18e04f4037d25 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:17:33.510047 359 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83b05a51148d47e49ea18e04f4037d25 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:17:33.510140 359 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 83b05a51148d47e49ea18e04f4037d25 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:17:33.511209 359 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83b05a51148d47e49ea18e04f4037d25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83b05a51148d47e49ea18e04f4037d25" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 36003 } }
I20260809 02:17:33.511570 359 leader_election.cc:302] T 00000000000000000000000000000000 P 83b05a51148d47e49ea18e04f4037d25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 83b05a51148d47e49ea18e04f4037d25; no voters:
I20260809 02:17:33.511905 359 leader_election.cc:288] T 00000000000000000000000000000000 P 83b05a51148d47e49ea18e04f4037d25 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:17:33.512020 368 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 83b05a51148d47e49ea18e04f4037d25 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:17:33.512285 368 raft_consensus.cc:686] T 00000000000000000000000000000000 P 83b05a51148d47e49ea18e04f4037d25 [term 1 LEADER]: Becoming Leader. State: Replica: 83b05a51148d47e49ea18e04f4037d25, State: Running, Role: LEADER
I20260809 02:17:33.512629 368 consensus_queue.cc:237] T 00000000000000000000000000000000 P 83b05a51148d47e49ea18e04f4037d25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83b05a51148d47e49ea18e04f4037d25" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 36003 } }
I20260809 02:17:33.513166 359 sys_catalog.cc:564] T 00000000000000000000000000000000 P 83b05a51148d47e49ea18e04f4037d25 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:17:33.513892 368 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83b05a51148d47e49ea18e04f4037d25 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 83b05a51148d47e49ea18e04f4037d25. Latest consensus state: current_term: 1 leader_uuid: "83b05a51148d47e49ea18e04f4037d25" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83b05a51148d47e49ea18e04f4037d25" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 36003 } } }
I20260809 02:17:33.514097 368 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83b05a51148d47e49ea18e04f4037d25 [sys.catalog]: This master's current role is: LEADER
I20260809 02:17:33.514858 373 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83b05a51148d47e49ea18e04f4037d25 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "83b05a51148d47e49ea18e04f4037d25" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83b05a51148d47e49ea18e04f4037d25" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 36003 } } }
I20260809 02:17:33.515014 373 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83b05a51148d47e49ea18e04f4037d25 [sys.catalog]: This master's current role is: LEADER
I20260809 02:17:33.515159 382 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:17:33.515350 359 ranger_client.cc:318] Using new properties file: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 02:17:33.522157 382 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:17:33.534098 382 catalog_manager.cc:1347] Generated new cluster ID: 9034a381061b411f9b30a5a6d7e963ff
I20260809 02:17:33.534173 382 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:17:33.546799 382 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:17:33.548125 382 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:17:33.557585 382 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 83b05a51148d47e49ea18e04f4037d25: Generated new TSK 0
I20260809 02:17:33.558292 382 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.75.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab.
Entry for principal kudu/127.15.75.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab.
I20260809 02:17:34.303797 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.75.148:39187/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.129:0
--local_ip_for_outbound_sockets=127.15.75.129
--webserver_interface=127.15.75.129
--webserver_port=0
--tserver_master_addrs=127.15.75.190:36003
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab
--principal=kudu/127.15.75.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:40839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:17:34.455191 429 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:17:34.455442 429 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:17:34.455487 429 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:17:34.460820 429 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:17:34.460902 429 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:17:34.460988 429 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.129
I20260809 02:17:34.468240 429 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:40839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.15.75.148:39187/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab
--principal=kudu/127.15.75.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.75.129
--webserver_port=0
--tserver_master_addrs=127.15.75.190:36003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.429
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.129
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:17:34.469422 429 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:17:34.470451 429 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:17:34.482543 444 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:17:34.482703 438 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:17:34.483048 429 server_base.cc:1034] running on GCE node
W20260809 02:17:34.483098 441 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:17:34.483510 429 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:17:34.484252 429 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:17:34.485438 429 hybrid_clock.cc:648] HybridClock initialized: now 1786241854485414 us; error 38 us; skew 500 ppm
Aug 09 02:17:34 dist-test-slave-s11t krb5kdc[32030](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241854, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:17:34.488947 429 init.cc:377] Logged in from keytab as kudu/127.15.75.129@KRBTEST.COM (short username kudu)
I20260809 02:17:34.490260 429 webserver.cc:460] Webserver started at http://127.15.75.129:41365/ using document root <none> and password file <none>
I20260809 02:17:34.491014 429 fs_manager.cc:359] Metadata directory not provided
I20260809 02:17:34.491118 429 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:17:34.491379 429 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 02:17:34 dist-test-slave-s11t krb5kdc[32030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241854, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.129@KRBTEST.COM for HTTP/127.15.75.148@KRBTEST.COM
I20260809 02:17:34.519605 429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/ts-0/data/instance:
uuid: "1ef5c582acd14ddba843db8c1e06fc8e"
format_stamp: "Formatted at 2026-08-09 02:17:34 on dist-test-slave-s11t"
server_key: "f70819b9f533a837e2cb9b1474c66d27"
server_key_iv: "a83125c298b2ead0dfb7d505f1511d97"
server_key_version: "kuduclusterkey@0"
I20260809 02:17:34.520525 429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/ts-0/wal/instance:
uuid: "1ef5c582acd14ddba843db8c1e06fc8e"
format_stamp: "Formatted at 2026-08-09 02:17:34 on dist-test-slave-s11t"
server_key: "f70819b9f533a837e2cb9b1474c66d27"
server_key_iv: "a83125c298b2ead0dfb7d505f1511d97"
server_key_version: "kuduclusterkey@0"
I20260809 02:17:34.534878 429 fs_manager.cc:691] Time spent creating directory manager: real 0.014s user 0.003s sys 0.000s
I20260809 02:17:34.550872 459 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:17:34.552675 429 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 02:17:34.552922 429 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/ts-0/wal
uuid: "1ef5c582acd14ddba843db8c1e06fc8e"
format_stamp: "Formatted at 2026-08-09 02:17:34 on dist-test-slave-s11t"
server_key: "f70819b9f533a837e2cb9b1474c66d27"
server_key_iv: "a83125c298b2ead0dfb7d505f1511d97"
server_key_version: "kuduclusterkey@0"
I20260809 02:17:34.553144 429 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:17:34.573899 429 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:17:34.577747 429 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:17:34.578001 429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:17:34.578805 429 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:17:34.579901 429 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:17:34.580011 429 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:17:34.580145 429 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:17:34.580241 429 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:17:34.591750 429 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.129:46373
I20260809 02:17:34.592662 429 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
I20260809 02:17:34.592969 621 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.129:46373 every 8 connection(s)
I20260809 02:17:34.595882 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 429
I20260809 02:17:34.596025 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/ts-0/wal/instance
I20260809 02:17:34.596344 15662 external_mini_cluster.cc:1428] Setting key dd223393df19821dc8e1b13e5eec470d
Aug 09 02:17:34 dist-test-slave-s11t krb5kdc[32030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241854, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.129@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.75.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.130@KRBTEST.COM" created.
I20260809 02:17:34.614852 625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:34.595085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.129:47111 (local address 127.15.75.190:36003)
0809 02:17:34.595333 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:34.595338 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:34.597744 (+ 2406us) server_negotiation.cc:407] Connection header received
0809 02:17:34.601400 (+ 3656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:34.601406 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:34.601475 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:34.601546 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:17:34.603354 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:34.604285 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:34.606219 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:34.606624 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:34.606707 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:34.609966 (+ 3259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:17:34.609973 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:17:34.609978 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:17:34.610012 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:17:34.611894 (+ 1882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:34.612544 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:34.612551 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:34.612556 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:34.612657 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:34.613212 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:34.613218 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:34.613223 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:34.613484 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:17:34.613576 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:34.614419 (+ 843us) server_negotiation.cc:299] Negotiation successful
0809 02:17:34.614579 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":68,"threads_started":1}
I20260809 02:17:34.616621 626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:34.595540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36003 (local address 127.15.75.129:47111)
0809 02:17:34.597576 (+ 2036us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:34.597639 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0809 02:17:34.601129 (+ 3490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:17:34.601992 (+ 863us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:17:34.602004 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:17:34.602380 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:17:34.603158 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:34.603183 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:34.604418 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:34.604424 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:34.606035 (+ 1611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:34.606049 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:34.606938 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:34.606942 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:34.607173 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:34.607935 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:17:34.607962 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:17:34.609747 (+ 1785us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:17:34.612020 (+ 2273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:17:34.612026 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:17:34.612042 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:17:34.612422 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:17:34.612815 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:17:34.612819 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:17:34.612822 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:17:34.613094 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:17:34.613649 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:17:34.613658 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:17:34.614108 (+ 450us) client_negotiation.cc:769] Sending connection context
0809 02:17:34.615055 (+ 947us) client_negotiation.cc:240] Negotiation successful
0809 02:17:34.615491 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1332,"thread_start_us":146,"threads_started":1}
I20260809 02:17:34.618346 623 heartbeater.cc:347] Connected to a master server at 127.15.75.190:36003
I20260809 02:17:34.618631 623 heartbeater.cc:464] Registering TS with master...
I20260809 02:17:34.619285 623 heartbeater.cc:511] Master 127.15.75.190:36003 requested a full tablet report, sending...
I20260809 02:17:34.621182 308 ts_manager.cc:194] Registered new tserver with Master: 1ef5c582acd14ddba843db8c1e06fc8e (127.15.75.129:46373)
I20260809 02:17:34.622848 308 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.75.129@KRBTEST.COM'} at 127.15.75.129:47111
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab.
Entry for principal kudu/127.15.75.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab.
I20260809 02:17:34.631734 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.75.148:39187/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.130:0
--local_ip_for_outbound_sockets=127.15.75.130
--webserver_interface=127.15.75.130
--webserver_port=0
--tserver_master_addrs=127.15.75.190:36003
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab
--principal=kudu/127.15.75.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:40839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 02:17:34 dist-test-slave-s11t krb5kdc[32030](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786241854, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260809 02:17:34.783177 630 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:17:34.783630 630 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:17:34.783749 630 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:17:34.789898 630 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:17:34.789978 630 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:17:34.790068 630 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.130
I20260809 02:17:34.797720 630 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:40839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.15.75.148:39187/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab
--principal=kudu/127.15.75.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.75.130
--webserver_port=0
--tserver_master_addrs=127.15.75.190:36003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.630
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.130
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:17:34.799058 630 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:17:34.800392 630 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:17:34.812880 643 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:17:34.812892 637 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:17:34.813656 638 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:17:34.814595 630 server_base.cc:1034] running on GCE node
I20260809 02:17:34.815208 630 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:17:34.816066 630 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:17:34.817286 630 hybrid_clock.cc:648] HybridClock initialized: now 1786241854817269 us; error 40 us; skew 500 ppm
Aug 09 02:17:34 dist-test-slave-s11t krb5kdc[32030](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241854, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:17:34.821125 630 init.cc:377] Logged in from keytab as kudu/127.15.75.130@KRBTEST.COM (short username kudu)
I20260809 02:17:34.822409 630 webserver.cc:460] Webserver started at http://127.15.75.130:38111/ using document root <none> and password file <none>
I20260809 02:17:34.823199 630 fs_manager.cc:359] Metadata directory not provided
I20260809 02:17:34.823302 630 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:17:34.823575 630 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 02:17:34 dist-test-slave-s11t krb5kdc[32030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241854, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.130@KRBTEST.COM for HTTP/127.15.75.148@KRBTEST.COM
I20260809 02:17:34.840896 630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/ts-1/data/instance:
uuid: "835786bd1e6240b5b80f7c2e969bf3d7"
format_stamp: "Formatted at 2026-08-09 02:17:34 on dist-test-slave-s11t"
server_key: "9e3fde1c171a16180f6a56446e10002c"
server_key_iv: "9713debaeea0c4a9f45ed76506c9232e"
server_key_version: "kuduclusterkey@0"
I20260809 02:17:34.841777 630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/ts-1/wal/instance:
uuid: "835786bd1e6240b5b80f7c2e969bf3d7"
format_stamp: "Formatted at 2026-08-09 02:17:34 on dist-test-slave-s11t"
server_key: "9e3fde1c171a16180f6a56446e10002c"
server_key_iv: "9713debaeea0c4a9f45ed76506c9232e"
server_key_version: "kuduclusterkey@0"
I20260809 02:17:34.858367 630 fs_manager.cc:691] Time spent creating directory manager: real 0.016s user 0.005s sys 0.000s
I20260809 02:17:34.882126 652 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:17:34.885524 630 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.004s sys 0.000s
I20260809 02:17:34.885767 630 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/ts-1/wal
uuid: "835786bd1e6240b5b80f7c2e969bf3d7"
format_stamp: "Formatted at 2026-08-09 02:17:34 on dist-test-slave-s11t"
server_key: "9e3fde1c171a16180f6a56446e10002c"
server_key_iv: "9713debaeea0c4a9f45ed76506c9232e"
server_key_version: "kuduclusterkey@0"
I20260809 02:17:34.885995 630 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:17:34.917860 630 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:17:34.921685 630 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:17:34.921914 630 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:17:34.922595 630 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:17:34.923599 630 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:17:34.923674 630 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:17:34.923745 630 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:17:34.923779 630 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:17:34.935354 630 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.130:35809
I20260809 02:17:34.936520 630 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
I20260809 02:17:34.936841 821 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.130:35809 every 8 connection(s)
I20260809 02:17:34.940932 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 630
I20260809 02:17:34.941074 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/ts-1/wal/instance
I20260809 02:17:34.941411 15662 external_mini_cluster.cc:1428] Setting key b415f4363d303c3225407c6e443a2a06
Aug 09 02:17:34 dist-test-slave-s11t krb5kdc[32030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241854, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.130@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:17:34.957296 625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:34.941905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.130:49391 (local address 127.15.75.190:36003)
0809 02:17:34.942291 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:34.942296 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:34.942316 (+ 20us) server_negotiation.cc:407] Connection header received
0809 02:17:34.943558 (+ 1242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:34.943562 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:34.943611 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:34.943661 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:17:34.945271 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:34.946093 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:34.947804 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:34.948163 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:34.948235 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:34.951180 (+ 2945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:17:34.951184 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:17:34.951187 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:17:34.951215 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:17:34.954661 (+ 3446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:34.955321 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:34.955325 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:34.955327 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:34.955425 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:34.955892 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:34.955896 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:34.955897 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:34.956130 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:17:34.956223 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:34.956969 (+ 746us) server_negotiation.cc:299] Negotiation successful
0809 02:17:34.957123 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292}
I20260809 02:17:34.959163 825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:34.939573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36003 (local address 127.15.75.130:49391)
0809 02:17:34.942030 (+ 2457us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:34.942089 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0809 02:17:34.943309 (+ 1220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:17:34.944073 (+ 764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:17:34.944081 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:17:34.944438 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:17:34.945101 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:34.945119 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:34.946254 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:34.946257 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:34.947645 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:34.947657 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:34.948430 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:34.948433 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:34.948619 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:34.949312 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:17:34.949333 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:17:34.951012 (+ 1679us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:17:34.954798 (+ 3786us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:17:34.954805 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:17:34.954821 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:17:34.955186 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:17:34.955547 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:17:34.955551 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:17:34.955553 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:17:34.955747 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:17:34.956297 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:17:34.956304 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:17:34.956617 (+ 313us) client_negotiation.cc:769] Sending connection context
0809 02:17:34.957744 (+ 1127us) client_negotiation.cc:240] Negotiation successful
0809 02:17:34.958102 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2219,"thread_start_us":169,"threads_started":1}
WARNING: no policy specified for kudu/127.15.75.131@KRBTEST.COM; defaulting to no policy
I20260809 02:17:34.960798 822 heartbeater.cc:347] Connected to a master server at 127.15.75.190:36003
I20260809 02:17:34.961051 822 heartbeater.cc:464] Registering TS with master...
I20260809 02:17:34.961925 822 heartbeater.cc:511] Master 127.15.75.190:36003 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.131@KRBTEST.COM" created.
I20260809 02:17:34.963563 308 ts_manager.cc:194] Registered new tserver with Master: 835786bd1e6240b5b80f7c2e969bf3d7 (127.15.75.130:35809)
I20260809 02:17:34.964320 308 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.75.130@KRBTEST.COM'} at 127.15.75.130:49391
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab.
Entry for principal kudu/127.15.75.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab.
I20260809 02:17:34.980600 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.75.148:39187/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.131:0
--local_ip_for_outbound_sockets=127.15.75.131
--webserver_interface=127.15.75.131
--webserver_port=0
--tserver_master_addrs=127.15.75.190:36003
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab
--principal=kudu/127.15.75.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:40839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 02:17:35 dist-test-slave-s11t krb5kdc[32030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241854, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.190@KRBTEST.COM for HTTP/127.15.75.148@KRBTEST.COM
W20260809 02:17:35.145802 832 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:17:35.146152 832 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:17:35.146210 832 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:17:35.152036 832 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:17:35.152119 832 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:17:35.152202 832 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.131
I20260809 02:17:35.159730 832 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:40839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.15.75.148:39187/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab
--principal=kudu/127.15.75.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.75.131
--webserver_port=0
--tserver_master_addrs=127.15.75.190:36003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.832
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.131
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:17:35.161101 832 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:17:35.162385 832 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:17:35.174824 839 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:17:35.177860 843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:17:35.178236 840 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:17:35.185775 832 server_base.cc:1034] running on GCE node
I20260809 02:17:35.186486 832 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:17:35.187341 832 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:17:35.188573 832 hybrid_clock.cc:648] HybridClock initialized: now 1786241855188506 us; error 94 us; skew 500 ppm
Aug 09 02:17:35 dist-test-slave-s11t krb5kdc[32030](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241855, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:17:35.192270 832 init.cc:377] Logged in from keytab as kudu/127.15.75.131@KRBTEST.COM (short username kudu)
I20260809 02:17:35.193439 832 webserver.cc:460] Webserver started at http://127.15.75.131:38551/ using document root <none> and password file <none>
I20260809 02:17:35.194170 832 fs_manager.cc:359] Metadata directory not provided
I20260809 02:17:35.194260 832 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:17:35.194509 832 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 02:17:35 dist-test-slave-s11t krb5kdc[32030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241855, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.131@KRBTEST.COM for HTTP/127.15.75.148@KRBTEST.COM
I20260809 02:17:35.231779 832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/ts-2/data/instance:
uuid: "2bc6bfe99f5941f6a328e6e37da65b07"
format_stamp: "Formatted at 2026-08-09 02:17:35 on dist-test-slave-s11t"
server_key: "8c26e26de206f32523ff2e74ae9df4d6"
server_key_iv: "eee0ab6e8630bf6af6fc7b220d004fc5"
server_key_version: "kuduclusterkey@0"
I20260809 02:17:35.232589 832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/ts-2/wal/instance:
uuid: "2bc6bfe99f5941f6a328e6e37da65b07"
format_stamp: "Formatted at 2026-08-09 02:17:35 on dist-test-slave-s11t"
server_key: "8c26e26de206f32523ff2e74ae9df4d6"
server_key_iv: "eee0ab6e8630bf6af6fc7b220d004fc5"
server_key_version: "kuduclusterkey@0"
I20260809 02:17:35.246202 832 fs_manager.cc:691] Time spent creating directory manager: real 0.013s user 0.004s sys 0.000s
I20260809 02:17:35.269676 854 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:17:35.271405 832 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 02:17:35.271545 832 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/ts-2/wal
uuid: "2bc6bfe99f5941f6a328e6e37da65b07"
format_stamp: "Formatted at 2026-08-09 02:17:35 on dist-test-slave-s11t"
server_key: "8c26e26de206f32523ff2e74ae9df4d6"
server_key_iv: "eee0ab6e8630bf6af6fc7b220d004fc5"
server_key_version: "kuduclusterkey@0"
I20260809 02:17:35.271682 832 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:17:35.312603 832 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:17:35.316254 832 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:17:35.316447 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:17:35.317175 832 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:17:35.318252 832 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:17:35.318367 832 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:17:35.318468 832 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:17:35.318532 832 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:17:35.332237 832 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.131:39483
I20260809 02:17:35.333479 832 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
I20260809 02:17:35.333724 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 832
I20260809 02:17:35.333871 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/ts-2/wal/instance
I20260809 02:17:35.334244 15662 external_mini_cluster.cc:1428] Setting key a60cc847c82cd90f09d5045e84b7defc
I20260809 02:17:35.334468 1030 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.131:39483 every 8 connection(s)
Aug 09 02:17:35 dist-test-slave-s11t krb5kdc[32030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241855, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.131@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:17:35.353724 625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:35.339900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.131:50067 (local address 127.15.75.190:36003)
0809 02:17:35.340019 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:35.340024 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:35.340144 (+ 120us) server_negotiation.cc:407] Connection header received
0809 02:17:35.341474 (+ 1330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:35.341478 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:35.341533 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:35.341584 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:17:35.343369 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:35.344213 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:35.346073 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:35.346433 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:35.346517 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:35.349687 (+ 3170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:17:35.349691 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:17:35.349694 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:17:35.349722 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:17:35.351393 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:35.351992 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:35.351996 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:35.351998 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:35.352082 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:35.352463 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:35.352467 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:35.352469 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:35.352673 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:17:35.352798 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:35.353442 (+ 644us) server_negotiation.cc:299] Negotiation successful
0809 02:17:35.353566 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 02:17:35.355697 1037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:35.338885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36003 (local address 127.15.75.131:50067)
0809 02:17:35.339969 (+ 1084us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:35.340027 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 02:17:35.341218 (+ 1191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:17:35.342006 (+ 788us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:17:35.342015 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:17:35.342460 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:17:35.343190 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:35.343212 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:35.344493 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:35.344497 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:35.345924 (+ 1427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:35.345936 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:35.346709 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:35.346713 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:35.346908 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:35.347648 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:17:35.347671 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:17:35.349467 (+ 1796us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:17:35.351523 (+ 2056us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:17:35.351529 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:17:35.351545 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:17:35.351879 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:17:35.352180 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:17:35.352183 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:17:35.352185 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:17:35.352356 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:17:35.352864 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:17:35.352870 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:17:35.353172 (+ 302us) client_negotiation.cc:769] Sending connection context
0809 02:17:35.354368 (+ 1196us) client_negotiation.cc:240] Negotiation successful
0809 02:17:35.354700 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":840,"thread_start_us":153,"threads_started":1}
I20260809 02:17:35.357085 1034 heartbeater.cc:347] Connected to a master server at 127.15.75.190:36003
I20260809 02:17:35.357321 1034 heartbeater.cc:464] Registering TS with master...
I20260809 02:17:35.357887 1034 heartbeater.cc:511] Master 127.15.75.190:36003 requested a full tablet report, sending...
I20260809 02:17:35.359020 308 ts_manager.cc:194] Registered new tserver with Master: 2bc6bfe99f5941f6a328e6e37da65b07 (127.15.75.131:39483)
I20260809 02:17:35.359736 308 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.75.131@KRBTEST.COM'} at 127.15.75.131:50067
I20260809 02:17:35.363632 15662 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 02:17:35 dist-test-slave-s11t krb5kdc[32030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241854, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.190@KRBTEST.COM for HTTP/127.15.75.148@KRBTEST.COM
I20260809 02:17:35.375854 625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:35.365886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40466 (local address 127.15.75.190:36003)
0809 02:17:35.365998 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:35.366002 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:35.366585 (+ 583us) server_negotiation.cc:407] Connection header received
0809 02:17:35.366784 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:35.366787 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:35.366833 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:35.366882 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:17:35.367883 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:35.368662 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:35.369841 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:35.370178 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:35.370262 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:35.372058 (+ 1796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:17:35.372062 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:17:35.372064 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:17:35.372091 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:17:35.373847 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:35.374357 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:35.374361 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:35.374363 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:35.374456 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:35.374784 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:35.374787 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:35.374789 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:35.375024 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:17:35.375129 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:35.375528 (+ 399us) server_negotiation.cc:299] Negotiation successful
0809 02:17:35.375657 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 02:17:35.379544 308 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40466:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:17:35.384016 308 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:17:35.411154 1057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:35.397052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.131:39483 (local address 127.0.0.1:37878)
0809 02:17:35.398265 (+ 1213us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:35.398277 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:17:35.398350 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:17:35.406556 (+ 8206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:17:35.406559 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:17:35.406577 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:17:35.406672 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:35.406679 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:35.408041 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:35.408044 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:35.409175 (+ 1131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:35.409190 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:35.410549 (+ 1359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:35.410553 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:35.410689 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:35.410733 (+ 44us) client_negotiation.cc:769] Sending connection context
0809 02:17:35.410858 (+ 125us) client_negotiation.cc:240] Negotiation successful
0809 02:17:35.410988 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1142,"mutex_wait_us":1,"thread_start_us":82,"threads_started":1}
I20260809 02:17:35.411470 1055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:35.396018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.129:46373 (local address 127.0.0.1:46588)
0809 02:17:35.397688 (+ 1670us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:35.397736 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 02:17:35.397862 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:17:35.399304 (+ 1442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:17:35.399310 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:17:35.399332 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:17:35.399478 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:35.399485 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:35.406411 (+ 6926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:35.406415 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:35.407797 (+ 1382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:35.407808 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:35.410575 (+ 2767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:35.410579 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:35.411150 (+ 571us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:35.411174 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 02:17:35.411246 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 02:17:35.411321 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":683,"mutex_wait_us":445,"thread_start_us":101,"threads_started":1}
I20260809 02:17:35.411631 1060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:35.397518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37878 (local address 127.15.75.131:39483)
0809 02:17:35.405827 (+ 8309us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:35.405846 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:35.405874 (+ 28us) server_negotiation.cc:407] Connection header received
0809 02:17:35.405960 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:35.405969 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:35.406210 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:35.406392 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:17:35.406819 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:35.407900 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:35.409382 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:35.410193 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:35.410319 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:35.410476 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:35.411383 (+ 907us) server_negotiation.cc:299] Negotiation successful
0809 02:17:35.411487 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8155,"thread_start_us":149,"threads_started":1}
I20260809 02:17:35.412240 1056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:35.396951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46588 (local address 127.15.75.129:46373)
0809 02:17:35.398824 (+ 1873us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:35.398831 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:35.398856 (+ 25us) server_negotiation.cc:407] Connection header received
0809 02:17:35.398930 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:35.398935 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:35.399130 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:35.399629 (+ 499us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:17:35.399757 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:35.405147 (+ 5390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:35.407948 (+ 2801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:35.410271 (+ 2323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:35.410376 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:35.410512 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:35.411988 (+ 1476us) server_negotiation.cc:299] Negotiation successful
0809 02:17:35.412093 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1739,"thread_start_us":129,"threads_started":1}
I20260809 02:17:35.416028 933 tablet_service.cc:1467] Processing CreateTablet for tablet 1486290ed8614ca78b2983ad393c26c7 (DEFAULT_TABLE table=test-table [id=8a359f62798a4e56bc0dea8d48fb003e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:17:35.417412 933 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1486290ed8614ca78b2983ad393c26c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:17:35.419154 521 tablet_service.cc:1467] Processing CreateTablet for tablet 1486290ed8614ca78b2983ad393c26c7 (DEFAULT_TABLE table=test-table [id=8a359f62798a4e56bc0dea8d48fb003e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:17:35.420204 521 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1486290ed8614ca78b2983ad393c26c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:17:35.425771 1062 tablet_bootstrap.cc:492] T 1486290ed8614ca78b2983ad393c26c7 P 2bc6bfe99f5941f6a328e6e37da65b07: Bootstrap starting.
I20260809 02:17:35.428203 1062 tablet_bootstrap.cc:654] T 1486290ed8614ca78b2983ad393c26c7 P 2bc6bfe99f5941f6a328e6e37da65b07: Neither blocks nor log segments found. Creating new log.
I20260809 02:17:35.429150 1062 log.cc:824] T 1486290ed8614ca78b2983ad393c26c7 P 2bc6bfe99f5941f6a328e6e37da65b07: Log is configured to *not* fsync() on all Append() calls
I20260809 02:17:35.430994 1065 tablet_bootstrap.cc:492] T 1486290ed8614ca78b2983ad393c26c7 P 1ef5c582acd14ddba843db8c1e06fc8e: Bootstrap starting.
I20260809 02:17:35.434340 1062 tablet_bootstrap.cc:492] T 1486290ed8614ca78b2983ad393c26c7 P 2bc6bfe99f5941f6a328e6e37da65b07: No bootstrap required, opened a new log
I20260809 02:17:35.435182 1065 tablet_bootstrap.cc:654] T 1486290ed8614ca78b2983ad393c26c7 P 1ef5c582acd14ddba843db8c1e06fc8e: Neither blocks nor log segments found. Creating new log.
I20260809 02:17:35.435431 1062 ts_tablet_manager.cc:1397] T 1486290ed8614ca78b2983ad393c26c7 P 2bc6bfe99f5941f6a328e6e37da65b07: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260809 02:17:35.435760 1059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:35.397453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.130:35809 (local address 127.0.0.1:42266)
0809 02:17:35.405418 (+ 7965us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:35.405432 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:17:35.405524 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:17:35.424907 (+ 19383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:17:35.424913 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:17:35.424944 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:17:35.425094 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:35.425103 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:35.432367 (+ 7264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:35.432372 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:35.433612 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:35.433625 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:35.435342 (+ 1717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:35.435346 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:35.435442 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:35.435470 (+ 28us) client_negotiation.cc:769] Sending connection context
0809 02:17:35.435542 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 02:17:35.435621 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7891,"thread_start_us":82,"threads_started":1}
I20260809 02:17:35.436110 1065 log.cc:824] T 1486290ed8614ca78b2983ad393c26c7 P 1ef5c582acd14ddba843db8c1e06fc8e: Log is configured to *not* fsync() on all Append() calls
I20260809 02:17:35.436439 1058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:35.397365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42266 (local address 127.15.75.130:35809)
0809 02:17:35.398566 (+ 1201us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:35.398581 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:35.410659 (+ 12078us) server_negotiation.cc:407] Connection header received
0809 02:17:35.410744 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:35.410750 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:35.410950 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:35.425324 (+ 14374us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:17:35.425517 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:35.432228 (+ 6711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:35.433796 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:35.434606 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:35.434715 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:35.434863 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:35.436181 (+ 1318us) server_negotiation.cc:299] Negotiation successful
0809 02:17:35.436301 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1096,"thread_start_us":121,"threads_started":1}
I20260809 02:17:35.439425 689 tablet_service.cc:1467] Processing CreateTablet for tablet 1486290ed8614ca78b2983ad393c26c7 (DEFAULT_TABLE table=test-table [id=8a359f62798a4e56bc0dea8d48fb003e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:17:35.440559 689 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1486290ed8614ca78b2983ad393c26c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:17:35.446292 1065 tablet_bootstrap.cc:492] T 1486290ed8614ca78b2983ad393c26c7 P 1ef5c582acd14ddba843db8c1e06fc8e: No bootstrap required, opened a new log
I20260809 02:17:35.446506 1065 ts_tablet_manager.cc:1397] T 1486290ed8614ca78b2983ad393c26c7 P 1ef5c582acd14ddba843db8c1e06fc8e: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.000s
I20260809 02:17:35.448119 1062 raft_consensus.cc:348] T 1486290ed8614ca78b2983ad393c26c7 P 2bc6bfe99f5941f6a328e6e37da65b07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ef5c582acd14ddba843db8c1e06fc8e" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 46373 } } peers { permanent_uuid: "2bc6bfe99f5941f6a328e6e37da65b07" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 39483 } } peers { permanent_uuid: "835786bd1e6240b5b80f7c2e969bf3d7" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 35809 } }
I20260809 02:17:35.451210 1065 raft_consensus.cc:348] T 1486290ed8614ca78b2983ad393c26c7 P 1ef5c582acd14ddba843db8c1e06fc8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ef5c582acd14ddba843db8c1e06fc8e" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 46373 } } peers { permanent_uuid: "2bc6bfe99f5941f6a328e6e37da65b07" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 39483 } } peers { permanent_uuid: "835786bd1e6240b5b80f7c2e969bf3d7" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 35809 } }
I20260809 02:17:35.451485 1065 raft_consensus.cc:374] T 1486290ed8614ca78b2983ad393c26c7 P 1ef5c582acd14ddba843db8c1e06fc8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:17:35.451587 1065 raft_consensus.cc:729] T 1486290ed8614ca78b2983ad393c26c7 P 1ef5c582acd14ddba843db8c1e06fc8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ef5c582acd14ddba843db8c1e06fc8e, State: Initialized, Role: FOLLOWER
I20260809 02:17:35.451601 1062 raft_consensus.cc:374] T 1486290ed8614ca78b2983ad393c26c7 P 2bc6bfe99f5941f6a328e6e37da65b07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:17:35.451774 1062 raft_consensus.cc:729] T 1486290ed8614ca78b2983ad393c26c7 P 2bc6bfe99f5941f6a328e6e37da65b07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2bc6bfe99f5941f6a328e6e37da65b07, State: Initialized, Role: FOLLOWER
I20260809 02:17:35.452085 1065 consensus_queue.cc:260] T 1486290ed8614ca78b2983ad393c26c7 P 1ef5c582acd14ddba843db8c1e06fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ef5c582acd14ddba843db8c1e06fc8e" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 46373 } } peers { permanent_uuid: "2bc6bfe99f5941f6a328e6e37da65b07" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 39483 } } peers { permanent_uuid: "835786bd1e6240b5b80f7c2e969bf3d7" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 35809 } }
I20260809 02:17:35.452428 1062 consensus_queue.cc:260] T 1486290ed8614ca78b2983ad393c26c7 P 2bc6bfe99f5941f6a328e6e37da65b07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ef5c582acd14ddba843db8c1e06fc8e" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 46373 } } peers { permanent_uuid: "2bc6bfe99f5941f6a328e6e37da65b07" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 39483 } } peers { permanent_uuid: "835786bd1e6240b5b80f7c2e969bf3d7" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 35809 } }
I20260809 02:17:35.453191 1065 ts_tablet_manager.cc:1428] T 1486290ed8614ca78b2983ad393c26c7 P 1ef5c582acd14ddba843db8c1e06fc8e: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260809 02:17:35.453501 1062 ts_tablet_manager.cc:1428] T 1486290ed8614ca78b2983ad393c26c7 P 2bc6bfe99f5941f6a328e6e37da65b07: Time spent starting tablet: real 0.018s user 0.006s sys 0.000s
I20260809 02:17:35.453584 623 heartbeater.cc:503] Master 127.15.75.190:36003 was elected leader, sending a full tablet report...
I20260809 02:17:35.453881 1034 heartbeater.cc:503] Master 127.15.75.190:36003 was elected leader, sending a full tablet report...
I20260809 02:17:35.457715 1071 tablet_bootstrap.cc:492] T 1486290ed8614ca78b2983ad393c26c7 P 835786bd1e6240b5b80f7c2e969bf3d7: Bootstrap starting.
I20260809 02:17:35.461488 1071 tablet_bootstrap.cc:654] T 1486290ed8614ca78b2983ad393c26c7 P 835786bd1e6240b5b80f7c2e969bf3d7: Neither blocks nor log segments found. Creating new log.
I20260809 02:17:35.463241 1071 log.cc:824] T 1486290ed8614ca78b2983ad393c26c7 P 835786bd1e6240b5b80f7c2e969bf3d7: Log is configured to *not* fsync() on all Append() calls
I20260809 02:17:35.466078 1071 tablet_bootstrap.cc:492] T 1486290ed8614ca78b2983ad393c26c7 P 835786bd1e6240b5b80f7c2e969bf3d7: No bootstrap required, opened a new log
I20260809 02:17:35.466300 1071 ts_tablet_manager.cc:1397] T 1486290ed8614ca78b2983ad393c26c7 P 835786bd1e6240b5b80f7c2e969bf3d7: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260809 02:17:35.469652 1071 raft_consensus.cc:348] T 1486290ed8614ca78b2983ad393c26c7 P 835786bd1e6240b5b80f7c2e969bf3d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ef5c582acd14ddba843db8c1e06fc8e" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 46373 } } peers { permanent_uuid: "2bc6bfe99f5941f6a328e6e37da65b07" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 39483 } } peers { permanent_uuid: "835786bd1e6240b5b80f7c2e969bf3d7" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 35809 } }
I20260809 02:17:35.469909 1071 raft_consensus.cc:374] T 1486290ed8614ca78b2983ad393c26c7 P 835786bd1e6240b5b80f7c2e969bf3d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:17:35.469972 1071 raft_consensus.cc:729] T 1486290ed8614ca78b2983ad393c26c7 P 835786bd1e6240b5b80f7c2e969bf3d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 835786bd1e6240b5b80f7c2e969bf3d7, State: Initialized, Role: FOLLOWER
I20260809 02:17:35.470585 1071 consensus_queue.cc:260] T 1486290ed8614ca78b2983ad393c26c7 P 835786bd1e6240b5b80f7c2e969bf3d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ef5c582acd14ddba843db8c1e06fc8e" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 46373 } } peers { permanent_uuid: "2bc6bfe99f5941f6a328e6e37da65b07" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 39483 } } peers { permanent_uuid: "835786bd1e6240b5b80f7c2e969bf3d7" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 35809 } }
I20260809 02:17:35.471325 822 heartbeater.cc:503] Master 127.15.75.190:36003 was elected leader, sending a full tablet report...
I20260809 02:17:35.471664 1071 ts_tablet_manager.cc:1428] T 1486290ed8614ca78b2983ad393c26c7 P 835786bd1e6240b5b80f7c2e969bf3d7: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
W20260809 02:17:35.587163 1036 tablet.cc:2369] T 1486290ed8614ca78b2983ad393c26c7 P 2bc6bfe99f5941f6a328e6e37da65b07: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 02:17:35.594620 624 tablet.cc:2369] T 1486290ed8614ca78b2983ad393c26c7 P 1ef5c582acd14ddba843db8c1e06fc8e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 02:17:35.695225 824 tablet.cc:2369] T 1486290ed8614ca78b2983ad393c26c7 P 835786bd1e6240b5b80f7c2e969bf3d7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:17:35.825354 1069 raft_consensus.cc:482] T 1486290ed8614ca78b2983ad393c26c7 P 2bc6bfe99f5941f6a328e6e37da65b07 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:17:35.825577 1069 raft_consensus.cc:504] T 1486290ed8614ca78b2983ad393c26c7 P 2bc6bfe99f5941f6a328e6e37da65b07 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ef5c582acd14ddba843db8c1e06fc8e" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 46373 } } peers { permanent_uuid: "2bc6bfe99f5941f6a328e6e37da65b07" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 39483 } } peers { permanent_uuid: "835786bd1e6240b5b80f7c2e969bf3d7" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 35809 } }
I20260809 02:17:35.826766 1069 leader_election.cc:288] T 1486290ed8614ca78b2983ad393c26c7 P 2bc6bfe99f5941f6a328e6e37da65b07 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1ef5c582acd14ddba843db8c1e06fc8e (127.15.75.129:46373), 835786bd1e6240b5b80f7c2e969bf3d7 (127.15.75.130:35809)
I20260809 02:17:35.832306 1037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:35.827151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.129:46373 (local address 127.15.75.131:36099)
0809 02:17:35.827328 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:35.827346 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 02:17:35.827465 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:17:35.827968 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:17:35.827974 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:17:35.827993 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:17:35.828125 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:35.828135 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:35.829369 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:35.829372 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:35.830714 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:35.830728 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:35.831839 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:35.831843 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:35.831953 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:35.831979 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 02:17:35.832058 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 02:17:35.832136 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58,"mutex_wait_us":33}
I20260809 02:17:35.832319 1056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:35.827448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.131:36099 (local address 127.15.75.129:46373)
0809 02:17:35.827617 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:35.827623 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:35.827640 (+ 17us) server_negotiation.cc:407] Connection header received
0809 02:17:35.827698 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:35.827702 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:35.827754 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:35.827833 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:17:35.828288 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:35.829253 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:35.831014 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:35.831698 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:35.831848 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:35.831906 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:35.832089 (+ 183us) server_negotiation.cc:299] Negotiation successful
0809 02:17:35.832173 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 02:17:35.832466 1058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:35.827909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.131:46867 (local address 127.15.75.130:35809)
0809 02:17:35.828047 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:35.828053 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:35.828073 (+ 20us) server_negotiation.cc:407] Connection header received
0809 02:17:35.828136 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:35.828140 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:35.828197 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:35.828291 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:17:35.828593 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:35.829433 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:35.831070 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:35.831750 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:35.831846 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:35.831902 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:35.832258 (+ 356us) server_negotiation.cc:299] Negotiation successful
0809 02:17:35.832347 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 02:17:35.832878 1083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:35.827454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.130:35809 (local address 127.15.75.131:46867)
0809 02:17:35.827841 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:35.827856 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:17:35.827946 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:17:35.828334 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:17:35.828336 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:17:35.828356 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:17:35.828459 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:35.828465 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:35.829556 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:35.829559 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:35.830929 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:35.830940 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:35.832016 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:35.832022 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:35.832115 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:35.832132 (+ 17us) client_negotiation.cc:769] Sending connection context
0809 02:17:35.832608 (+ 476us) client_negotiation.cc:240] Negotiation successful
0809 02:17:35.832678 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"mutex_wait_us":231,"thread_start_us":122,"threads_started":1}
I20260809 02:17:35.833133 556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1486290ed8614ca78b2983ad393c26c7" candidate_uuid: "2bc6bfe99f5941f6a328e6e37da65b07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ef5c582acd14ddba843db8c1e06fc8e" is_pre_election: true
I20260809 02:17:35.833479 556 raft_consensus.cc:2454] T 1486290ed8614ca78b2983ad393c26c7 P 1ef5c582acd14ddba843db8c1e06fc8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2bc6bfe99f5941f6a328e6e37da65b07 in term 0.
I20260809 02:17:35.833823 751 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1486290ed8614ca78b2983ad393c26c7" candidate_uuid: "2bc6bfe99f5941f6a328e6e37da65b07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "835786bd1e6240b5b80f7c2e969bf3d7" is_pre_election: true
I20260809 02:17:35.834105 751 raft_consensus.cc:2454] T 1486290ed8614ca78b2983ad393c26c7 P 835786bd1e6240b5b80f7c2e969bf3d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2bc6bfe99f5941f6a328e6e37da65b07 in term 0.
I20260809 02:17:35.834074 856 leader_election.cc:302] T 1486290ed8614ca78b2983ad393c26c7 P 2bc6bfe99f5941f6a328e6e37da65b07 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ef5c582acd14ddba843db8c1e06fc8e, 2bc6bfe99f5941f6a328e6e37da65b07; no voters:
I20260809 02:17:35.834400 1069 raft_consensus.cc:2781] T 1486290ed8614ca78b2983ad393c26c7 P 2bc6bfe99f5941f6a328e6e37da65b07 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:17:35.834508 1069 raft_consensus.cc:482] T 1486290ed8614ca78b2983ad393c26c7 P 2bc6bfe99f5941f6a328e6e37da65b07 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:17:35.834560 1069 raft_consensus.cc:3037] T 1486290ed8614ca78b2983ad393c26c7 P 2bc6bfe99f5941f6a328e6e37da65b07 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:17:35.836103 1069 raft_consensus.cc:504] T 1486290ed8614ca78b2983ad393c26c7 P 2bc6bfe99f5941f6a328e6e37da65b07 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ef5c582acd14ddba843db8c1e06fc8e" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 46373 } } peers { permanent_uuid: "2bc6bfe99f5941f6a328e6e37da65b07" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 39483 } } peers { permanent_uuid: "835786bd1e6240b5b80f7c2e969bf3d7" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 35809 } }
I20260809 02:17:35.836462 1069 leader_election.cc:288] T 1486290ed8614ca78b2983ad393c26c7 P 2bc6bfe99f5941f6a328e6e37da65b07 [CANDIDATE]: Term 1 election: Requested vote from peers 1ef5c582acd14ddba843db8c1e06fc8e (127.15.75.129:46373), 835786bd1e6240b5b80f7c2e969bf3d7 (127.15.75.130:35809)
I20260809 02:17:35.836923 556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1486290ed8614ca78b2983ad393c26c7" candidate_uuid: "2bc6bfe99f5941f6a328e6e37da65b07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ef5c582acd14ddba843db8c1e06fc8e"
I20260809 02:17:35.836930 751 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1486290ed8614ca78b2983ad393c26c7" candidate_uuid: "2bc6bfe99f5941f6a328e6e37da65b07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "835786bd1e6240b5b80f7c2e969bf3d7"
I20260809 02:17:35.837054 751 raft_consensus.cc:3037] T 1486290ed8614ca78b2983ad393c26c7 P 835786bd1e6240b5b80f7c2e969bf3d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:17:35.837054 556 raft_consensus.cc:3037] T 1486290ed8614ca78b2983ad393c26c7 P 1ef5c582acd14ddba843db8c1e06fc8e [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:17:35.838323 556 raft_consensus.cc:2454] T 1486290ed8614ca78b2983ad393c26c7 P 1ef5c582acd14ddba843db8c1e06fc8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2bc6bfe99f5941f6a328e6e37da65b07 in term 1.
I20260809 02:17:35.838323 751 raft_consensus.cc:2454] T 1486290ed8614ca78b2983ad393c26c7 P 835786bd1e6240b5b80f7c2e969bf3d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2bc6bfe99f5941f6a328e6e37da65b07 in term 1.
I20260809 02:17:35.838624 856 leader_election.cc:302] T 1486290ed8614ca78b2983ad393c26c7 P 2bc6bfe99f5941f6a328e6e37da65b07 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2bc6bfe99f5941f6a328e6e37da65b07, 835786bd1e6240b5b80f7c2e969bf3d7; no voters:
I20260809 02:17:35.838795 1069 raft_consensus.cc:2781] T 1486290ed8614ca78b2983ad393c26c7 P 2bc6bfe99f5941f6a328e6e37da65b07 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:17:35.839016 1069 raft_consensus.cc:686] T 1486290ed8614ca78b2983ad393c26c7 P 2bc6bfe99f5941f6a328e6e37da65b07 [term 1 LEADER]: Becoming Leader. State: Replica: 2bc6bfe99f5941f6a328e6e37da65b07, State: Running, Role: LEADER
I20260809 02:17:35.839394 1069 consensus_queue.cc:237] T 1486290ed8614ca78b2983ad393c26c7 P 2bc6bfe99f5941f6a328e6e37da65b07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ef5c582acd14ddba843db8c1e06fc8e" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 46373 } } peers { permanent_uuid: "2bc6bfe99f5941f6a328e6e37da65b07" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 39483 } } peers { permanent_uuid: "835786bd1e6240b5b80f7c2e969bf3d7" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 35809 } }
I20260809 02:17:35.842862 307 catalog_manager.cc:5495] T 1486290ed8614ca78b2983ad393c26c7 P 2bc6bfe99f5941f6a328e6e37da65b07 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2bc6bfe99f5941f6a328e6e37da65b07 (127.15.75.131). New cstate: current_term: 1 leader_uuid: "2bc6bfe99f5941f6a328e6e37da65b07" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ef5c582acd14ddba843db8c1e06fc8e" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 46373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2bc6bfe99f5941f6a328e6e37da65b07" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 39483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "835786bd1e6240b5b80f7c2e969bf3d7" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 35809 } health_report { overall_health: UNKNOWN } } }
I20260809 02:17:35.853541 1060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:35.850033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37894 (local address 127.15.75.131:39483)
0809 02:17:35.850144 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:35.850147 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:35.850161 (+ 14us) server_negotiation.cc:407] Connection header received
0809 02:17:35.850321 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:35.850323 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:35.850382 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:35.850466 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:17:35.850676 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:35.851160 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:35.852021 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:35.852234 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:35.852331 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:35.852620 (+ 289us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:17:35.853123 (+ 503us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:17:35.853219 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:35.853294 (+ 75us) server_negotiation.cc:299] Negotiation successful
0809 02:17:35.853342 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 02:17:35.859158 556 raft_consensus.cc:1261] T 1486290ed8614ca78b2983ad393c26c7 P 1ef5c582acd14ddba843db8c1e06fc8e [term 1 FOLLOWER]: Refusing update from remote peer 2bc6bfe99f5941f6a328e6e37da65b07: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:17:35.859158 751 raft_consensus.cc:1261] T 1486290ed8614ca78b2983ad393c26c7 P 835786bd1e6240b5b80f7c2e969bf3d7 [term 1 FOLLOWER]: Refusing update from remote peer 2bc6bfe99f5941f6a328e6e37da65b07: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:17:35.860013 1084 consensus_queue.cc:1035] T 1486290ed8614ca78b2983ad393c26c7 P 2bc6bfe99f5941f6a328e6e37da65b07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "835786bd1e6240b5b80f7c2e969bf3d7" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 35809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:17:35.860236 1069 consensus_queue.cc:1035] T 1486290ed8614ca78b2983ad393c26c7 P 2bc6bfe99f5941f6a328e6e37da65b07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ef5c582acd14ddba843db8c1e06fc8e" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 46373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:17:35.868911 1089 mvcc.cc:205] Tried to move back new op lower bound from 7316446641591951360 to 7316446641520652288. Current Snapshot: MvccSnapshot[applied={T|T < 7316446641591951360}]
I20260809 02:17:35.870584 1091 mvcc.cc:205] Tried to move back new op lower bound from 7316446641591951360 to 7316446641520652288. Current Snapshot: MvccSnapshot[applied={T|T < 7316446641591951360}]
I20260809 02:17:35.877368 305 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40466:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:17:35.877542 305 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40466:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:17:35.880030 305 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 83b05a51148d47e49ea18e04f4037d25: Sending DeleteTablet for 3 replicas of tablet 1486290ed8614ca78b2983ad393c26c7
I20260809 02:17:35.880777 933 tablet_service.cc:1514] Processing DeleteTablet for tablet 1486290ed8614ca78b2983ad393c26c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:17:35 UTC) from {username='kudu', principal='kudu/127.15.75.190@KRBTEST.COM'} at 127.0.0.1:37878
I20260809 02:17:35.880803 689 tablet_service.cc:1514] Processing DeleteTablet for tablet 1486290ed8614ca78b2983ad393c26c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:17:35 UTC) from {username='kudu', principal='kudu/127.15.75.190@KRBTEST.COM'} at 127.0.0.1:42266
I20260809 02:17:35.881286 1105 tablet_replica.cc:330] T 1486290ed8614ca78b2983ad393c26c7 P 2bc6bfe99f5941f6a328e6e37da65b07: stopping tablet replica
I20260809 02:17:35.881275 521 tablet_service.cc:1514] Processing DeleteTablet for tablet 1486290ed8614ca78b2983ad393c26c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:17:35 UTC) from {username='kudu', principal='kudu/127.15.75.190@KRBTEST.COM'} at 127.0.0.1:46588
I20260809 02:17:35.881600 1105 raft_consensus.cc:2229] T 1486290ed8614ca78b2983ad393c26c7 P 2bc6bfe99f5941f6a328e6e37da65b07 [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:17:35.881793 1109 tablet_replica.cc:330] T 1486290ed8614ca78b2983ad393c26c7 P 1ef5c582acd14ddba843db8c1e06fc8e: stopping tablet replica
I20260809 02:17:35.882014 1105 raft_consensus.cc:2258] T 1486290ed8614ca78b2983ad393c26c7 P 2bc6bfe99f5941f6a328e6e37da65b07 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:17:35.882189 1109 raft_consensus.cc:2229] T 1486290ed8614ca78b2983ad393c26c7 P 1ef5c582acd14ddba843db8c1e06fc8e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:17:35.882390 1107 tablet_replica.cc:330] T 1486290ed8614ca78b2983ad393c26c7 P 835786bd1e6240b5b80f7c2e969bf3d7: stopping tablet replica
I20260809 02:17:35.882637 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 429
I20260809 02:17:35.882674 1107 raft_consensus.cc:2229] T 1486290ed8614ca78b2983ad393c26c7 P 835786bd1e6240b5b80f7c2e969bf3d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:17:35.882961 1107 raft_consensus.cc:2258] T 1486290ed8614ca78b2983ad393c26c7 P 835786bd1e6240b5b80f7c2e969bf3d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:17:35.886541 1105 ts_tablet_manager.cc:1905] T 1486290ed8614ca78b2983ad393c26c7 P 2bc6bfe99f5941f6a328e6e37da65b07: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:17:35.889288 1107 ts_tablet_manager.cc:1905] T 1486290ed8614ca78b2983ad393c26c7 P 835786bd1e6240b5b80f7c2e969bf3d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:17:35.890434 1105 ts_tablet_manager.cc:1918] T 1486290ed8614ca78b2983ad393c26c7 P 2bc6bfe99f5941f6a328e6e37da65b07: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:17:35.890543 1105 log.cc:1196] T 1486290ed8614ca78b2983ad393c26c7 P 2bc6bfe99f5941f6a328e6e37da65b07: Deleting WAL directory at /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/ts-2/wal/wals/1486290ed8614ca78b2983ad393c26c7
I20260809 02:17:35.890873 1105 ts_tablet_manager.cc:1939] T 1486290ed8614ca78b2983ad393c26c7 P 2bc6bfe99f5941f6a328e6e37da65b07: Deleting consensus metadata
I20260809 02:17:35.892066 1107 ts_tablet_manager.cc:1918] T 1486290ed8614ca78b2983ad393c26c7 P 835786bd1e6240b5b80f7c2e969bf3d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:17:35.892184 1107 log.cc:1196] T 1486290ed8614ca78b2983ad393c26c7 P 835786bd1e6240b5b80f7c2e969bf3d7: Deleting WAL directory at /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241741788589-15662-0/minicluster-data/ts-1/wal/wals/1486290ed8614ca78b2983ad393c26c7
I20260809 02:17:35.892231 32751 catalog_manager.cc:4841] TS 2bc6bfe99f5941f6a328e6e37da65b07 (127.15.75.131:39483): tablet 1486290ed8614ca78b2983ad393c26c7 (table test-table [id=8a359f62798a4e56bc0dea8d48fb003e]) successfully deleted
I20260809 02:17:35.892529 1107 ts_tablet_manager.cc:1939] T 1486290ed8614ca78b2983ad393c26c7 P 835786bd1e6240b5b80f7c2e969bf3d7: Deleting consensus metadata
I20260809 02:17:35.893697 32747 catalog_manager.cc:4841] TS 835786bd1e6240b5b80f7c2e969bf3d7 (127.15.75.130:35809): tablet 1486290ed8614ca78b2983ad393c26c7 (table test-table [id=8a359f62798a4e56bc0dea8d48fb003e]) successfully deleted
W20260809 02:17:35.895629 32747 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.75.129:46373 (error 108)
I20260809 02:17:35.896458 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 630
I20260809 02:17:35.896638 1059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:35.896348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.129:46373 (local address 127.0.0.1:46590)
0809 02:17:35.896478 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:35.896558 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.129:46373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260809 02:17:35.902580 32747 catalog_manager.cc:4584] TS 1ef5c582acd14ddba843db8c1e06fc8e (127.15.75.129:46373): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1486290ed8614ca78b2983ad393c26c7: Network error: Client connection negotiation failed: client connection to 127.15.75.129:46373: connect: Connection refused (error 111)
I20260809 02:17:35.908277 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 832
I20260809 02:17:35.918780 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 32725
I20260809 02:17:35.935765 15662 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 02:17:36.280299 15662 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 02:17:36.280426 15662 mini_ranger.cc:66] Stopping Ranger...
I20260809 02:17:36.635524 15662 mini_ranger.cc:68] Stopped Ranger
2026-08-09 02:17:36.635 UTC [32080] LOG: received smart shutdown request
2026-08-09 02:17:36.639 UTC [32080] LOG: background worker "logical replication launcher" (PID 32089) exited with exit code 1
2026-08-09 02:17:36.640 UTC [32084] LOG: shutting down
2026-08-09 02:17:36.791 UTC [32080] LOG: database system is shut down
2026-08-09T02:17:36Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (48614 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:17:36 dist-test-slave-s11t krb5kdc[1124](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:17:36 dist-test-slave-s11t krb5kdc[1124](info): set up 2 sockets
Aug 09 02:17:36 dist-test-slave-s11t krb5kdc[1124](info): commencing operation
krb5kdc: starting...
W20260809 02:17:38.299561 15662 mini_kdc.cc:121] Time spent starting KDC: real 1.359s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Aug 09 02:17:38 dist-test-slave-s11t krb5kdc[1124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241858, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:17:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:17:38Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.15.75.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.15.75.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.15.75.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/krb5kdc/rangeradmin_127.15.75.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.15.75.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/krb5kdc/rangeradmin_127.15.75.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.15.75.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.15.75.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.15.75.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/krb5kdc/rangerlookup_127.15.75.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.15.75.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/krb5kdc/rangerlookup_127.15.75.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.15.75.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.15.75.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.15.75.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/krb5kdc/HTTP_127.15.75.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.15.75.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/krb5kdc/HTTP_127.15.75.148@KRBTEST.COM.keytab.
I20260809 02:17:38.476075 15662 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-tasktcWNU7/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/postgres -l logfile start
2026-08-09 02:17:40.785 UTC [1175] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 02:17:40.785 UTC [1175] LOG: listening on IPv4 address "127.15.75.148", port 59323
2026-08-09 02:17:40.788 UTC [1175] LOG: listening on Unix socket "/tmp/.s.PGSQL.59323"
2026-08-09 02:17:40.810 UTC [1182] LOG: database system was shut down at 2026-08-09 02:17:39 UTC
2026-08-09 02:17:40.813 UTC [1175] LOG: database system is ready to accept connections
I20260809 02:17:41.971752 15662 mini_postgres.cc:96] Postgres bound to 59323
2026-08-09 02:17:41.979 UTC [1192] FATAL: database "slave" does not exist
127.15.75.148:59323 - accepting connections
I20260809 02:17:41.980144 15662 mini_ranger.cc:161] Starting Ranger...
I20260809 02:17:41.996209 15662 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 02:17:42.418208 15662 mini_ranger.cc:87] Created ranger Postgres database
I20260809 02:17:42.418298 15662 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ranger-admin
2026-08-09 02:17:42,695 [I] DB FLAVOR :POSTGRES
2026-08-09 02:17:42,695 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 02:17:42,695 [I] Checking connection..
2026-08-09 02:17:42,695 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktcWNU7/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktcWNU7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.75.148:59323/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 02:17:42,936 [I] Checking connection passed.
2026-08-09 02:17:42,937 [I] --------- Verifying version history table ---------
2026-08-09 02:17:42,937 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktcWNU7/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktcWNU7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.75.148:59323/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 02:17:43,187 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 02:17:43,188 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktcWNU7/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktcWNU7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.75.148:59323/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 02:17:43,456 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 02:17:43,456 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 02:17:43,456 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktcWNU7/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktcWNU7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.75.148:59323/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasktcWNU7/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 02:17:43.709 UTC [1286] WARNING: there is no transaction in progress
2026-08-09 02:17:43,718 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 02:17:43,719 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktcWNU7/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktcWNU7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.75.148:59323/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 02:17:43,988 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 02:17:43,988 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 02:17:43,989 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktcWNU7/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktcWNU7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.75.148:59323/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 02:17:44,243 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktcWNU7/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktcWNU7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.75.148:59323/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 02:17:44,514 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktcWNU7/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktcWNU7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.75.148:59323/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-s11t.c.gcp-upstream.internal','N') ;"
2026-08-09 02:17:44,743 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktcWNU7/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktcWNU7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.75.148:59323/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 02:17:45,012 [I] Table x_portal_user does not exist in database ranger
2026-08-09 02:17:45,013 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktcWNU7/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktcWNU7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.75.148:59323/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 02:17:45,272 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 02:17:45,272 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktcWNU7/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktcWNU7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.75.148:59323/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 02:17:45,526 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktcWNU7/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktcWNU7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.75.148:59323/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 02:17:45,793 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 02:17:45,793 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktcWNU7/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktcWNU7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.75.148:59323/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasktcWNU7/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 02:17:46.692 UTC [1452] WARNING: there is no transaction in progress
2026-08-09 02:17:46.708 UTC [1452] WARNING: there is no transaction in progress
2026-08-09 02:17:46.723 UTC [1452] WARNING: there is no transaction in progress
2026-08-09 02:17:46.738 UTC [1452] WARNING: there is no transaction in progress
2026-08-09 02:17:46.753 UTC [1452] WARNING: there is no transaction in progress
2026-08-09 02:17:46.770 UTC [1452] WARNING: there is no transaction in progress
2026-08-09 02:17:46.823 UTC [1452] WARNING: there is no transaction in progress
2026-08-09 02:17:46.830 UTC [1452] WARNING: there is no transaction in progress
2026-08-09 02:17:46.845 UTC [1452] WARNING: there is no transaction in progress
2026-08-09 02:17:46.855 UTC [1452] WARNING: there is no transaction in progress
2026-08-09 02:17:46.866 UTC [1452] WARNING: there is no transaction in progress
2026-08-09 02:17:46.880 UTC [1452] WARNING: there is no transaction in progress
2026-08-09 02:17:46.889 UTC [1452] WARNING: there is no transaction in progress
2026-08-09 02:17:46.896 UTC [1452] WARNING: there is no transaction in progress
2026-08-09 02:17:46.902 UTC [1452] WARNING: there is no transaction in progress
2026-08-09 02:17:46.908 UTC [1452] WARNING: there is no transaction in progress
2026-08-09 02:17:47,508 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 02:17:47,509 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktcWNU7/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktcWNU7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.75.148:59323/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 02:17:47,767 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 02:17:47,768 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktcWNU7/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktcWNU7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.75.148:59323/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 02:17:48,036 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 02:17:48,036 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktcWNU7/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktcWNU7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.75.148:59323/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 02:17:48,300 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 02:17:48,301 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktcWNU7/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktcWNU7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.75.148:59323/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 02:17:48,548 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktcWNU7/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktcWNU7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.75.148:59323/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 02:17:48,808 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktcWNU7/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktcWNU7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.75.148:59323/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-s11t.c.gcp-upstream.internal';"
2026-08-09 02:17:49,063 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 02:17:49,064 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktcWNU7/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktcWNU7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.75.148:59323/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 02:17:49,317 [I] DB_PATCHES have already been applied
I20260809 02:17:49.323778 15662 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ranger-admin:/tmp/dist-test-tasktcWNU7/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-tasktcWNU7/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-tasktcWNU7/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktcWNU7/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 02:17:49.323846 15662 mini_ranger.cc:193] Using host: 127.15.75.148
I20260809 02:17:49.328286 15662 mini_ranger.cc:227] Ranger admin URL: http://127.15.75.148:35289
Aug 09, 2026 2:17:49 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 2:17:49 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 2:17:49 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 2:17:49 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 2:17:49 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 2:17:50 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 2:17:50 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 02:17:50.677096 15662 mini_ranger.cc:160] Time spent starting Ranger: real 8.697s user 0.000s sys 0.012s
Aug 09, 2026 2:18:11 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 2:18:11 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 09, 2026 2:18:11 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 2:18:11 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 2:18:11 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 2:18:11 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 2:18:11 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 2:18:11 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 2:18:11 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 2:18:11 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 2:18:11 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 2:18:11 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 2:18:11 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 2:18:11 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 2:18:11 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 2:18:11 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 2:18:11 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 2:18:12 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 02:18:12.826366 15662 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.15.75.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.15.75.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.15.75.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/krb5kdc/rangerkms_127.15.75.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.15.75.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/krb5kdc/rangerkms_127.15.75.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.15.75.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.15.75.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.15.75.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/krb5kdc/HTTP_127.15.75.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.15.75.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/krb5kdc/HTTP_127.15.75.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 02:18:12 dist-test-slave-s11t krb5kdc[1124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241892, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 02:18:12.917840 15662 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 02:18:12.928548 15662 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 02:18:13.765676 15662 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 02:18:13.765769 15662 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ranger-kms
I20260809 02:18:13.765794 15662 mini_ranger_kms.cc:227] Using postgres at 127.15.75.148:59323
2026-08-09 02:18:13,945 [I] DB FLAVOR :POSTGRES
2026-08-09 02:18:13,945 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 02:18:13,946 [I] Checking connection
2026-08-09 02:18:13,946 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktcWNU7/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.75.148:59323/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 02:18:14,195 [I] connection success
2026-08-09 02:18:14,196 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 02:18:14,196 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 02:18:14,196 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktcWNU7/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.75.148:59323/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 02:18:14,450 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 02:18:14,450 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 02:18:14,450 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 02:18:14,450 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktcWNU7/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.75.148:59323/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 02:18:14,707 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 02:18:15.210662 15662 mini_ranger_kms.cc:310] Created kms service
I20260809 02:18:15.409433 15662 mini_ranger_kms.cc:326] Created kudu user
I20260809 02:18:15.451929 15662 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 02:18:15 dist-test-slave-s11t krb5kdc[1124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241892, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.15.75.148@KRBTEST.COM
I20260809 02:18:15.844879 15662 mini_ranger_kms.cc:384] Added ranger policy
I20260809 02:18:15.845047 15662 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ranger-kms:/tmp/dist-test-tasktcWNU7/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktcWNU7/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-tasktcWNU7/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 02:18:15.845073 15662 mini_ranger_kms.cc:240] Using host: 127.15.75.148
I20260809 02:18:15.850289 15662 mini_ranger_kms.cc:276] Ranger KMS PID: 1727
I20260809 02:18:15.850373 15662 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.15.75.148:44415
Aug 09, 2026 2:18:16 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 2:18:16 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 2:18:16 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 2:18:16 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 2:18:17 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 2:18:17 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 09 02:18:18 dist-test-slave-s11t krb5kdc[1124](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786241898, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.75.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:18:18.573904 15662 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.656s user 0.010s sys 0.154s
I20260809 02:18:18.574041 15662 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 02:18:18.574124 15662 mini_ranger_kms.cc:401] 127.15.75.148:44415/kms/v1/keys
Aug 09 02:18:18 dist-test-slave-s11t krb5kdc[1124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241898, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.75.148@KRBTEST.COM for HTTP/127.15.75.148@KRBTEST.COM
Aug 09 02:18:19 dist-test-slave-s11t krb5kdc[1124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241898, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.75.148@KRBTEST.COM for HTTP/127.15.75.148@KRBTEST.COM
Aug 09 02:18:21 dist-test-slave-s11t krb5kdc[1124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241901, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.15.75.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab.
Entry for principal kudu/127.15.75.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab.
I20260809 02:18:21.332163 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:44007
--webserver_interface=127.15.75.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab
--principal=kudu/127.15.75.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:36263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.75.190:44007
--ranger_config_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.75.148:44415/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:18:21.445199 1820 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:18:21.445457 1820 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:18:21.445497 1820 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:18:21.449573 1820 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:18:21.449630 1820 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:18:21.449648 1820 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:18:21.449664 1820 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:18:21.449680 1820 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:18:21.455067 1820 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:36263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/master-0/wal
--ranger_kms_url=127.15.75.148:44415/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.15.75.190:44007
--ranger_config_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab
--principal=kudu/127.15.75.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:44007
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.75.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1820
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:18:21.456094 1820 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:18:21.457173 1820 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:18:21.466256 1829 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:21.466296 1820 server_base.cc:1034] running on GCE node
W20260809 02:18:21.466261 1830 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:21.466625 1833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:21.467128 1820 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:18:21.468202 1820 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:18:21.469376 1820 hybrid_clock.cc:648] HybridClock initialized: now 1786241901469352 us; error 41 us; skew 500 ppm
Aug 09 02:18:21 dist-test-slave-s11t krb5kdc[1124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241901, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:18:21.472122 1820 init.cc:377] Logged in from keytab as kudu/127.15.75.190@KRBTEST.COM (short username kudu)
I20260809 02:18:21.473129 1820 webserver.cc:460] Webserver started at http://127.15.75.190:32865/ using document root <none> and password file <none>
I20260809 02:18:21.473655 1820 fs_manager.cc:359] Metadata directory not provided
I20260809 02:18:21.473711 1820 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:18:21.473928 1820 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 02:18:21 dist-test-slave-s11t krb5kdc[1124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241901, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.190@KRBTEST.COM for HTTP/127.15.75.148@KRBTEST.COM
I20260809 02:18:21.527089 1820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/master-0/data/instance:
uuid: "61b8e6f14b0a46f99eb43653e7b9b450"
format_stamp: "Formatted at 2026-08-09 02:18:21 on dist-test-slave-s11t"
server_key: "706c37f8b5bb96f986565214f34e970f"
server_key_iv: "6cbd2ddec6f7ebb22972dbb774c6a5dc"
server_key_version: "kuduclusterkey@0"
I20260809 02:18:21.527695 1820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/master-0/wal/instance:
uuid: "61b8e6f14b0a46f99eb43653e7b9b450"
format_stamp: "Formatted at 2026-08-09 02:18:21 on dist-test-slave-s11t"
server_key: "706c37f8b5bb96f986565214f34e970f"
server_key_iv: "6cbd2ddec6f7ebb22972dbb774c6a5dc"
server_key_version: "kuduclusterkey@0"
I20260809 02:18:21.531755 1820 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 02:18:21.544989 1841 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:21.546336 1820 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 02:18:21.546496 1820 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/master-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/master-0/wal
uuid: "61b8e6f14b0a46f99eb43653e7b9b450"
format_stamp: "Formatted at 2026-08-09 02:18:21 on dist-test-slave-s11t"
server_key: "706c37f8b5bb96f986565214f34e970f"
server_key_iv: "6cbd2ddec6f7ebb22972dbb774c6a5dc"
server_key_version: "kuduclusterkey@0"
I20260809 02:18:21.546625 1820 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:18:21.570250 1820 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:18:21.573266 1820 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:18:21.573474 1820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:18:21.581704 1820 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.190:44007
I20260809 02:18:21.581718 1915 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.190:44007 every 8 connection(s)
I20260809 02:18:21.582669 1820 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
I20260809 02:18:21.585481 1916 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:18:21.589550 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 1820
I20260809 02:18:21.589675 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/master-0/wal/instance
I20260809 02:18:21.589973 15662 external_mini_cluster.cc:1428] Setting key 5a461dd29f91bcd3ac7c783ed964bd25
I20260809 02:18:21.591254 1916 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61b8e6f14b0a46f99eb43653e7b9b450: Bootstrap starting.
I20260809 02:18:21.593601 1916 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 61b8e6f14b0a46f99eb43653e7b9b450: Neither blocks nor log segments found. Creating new log.
I20260809 02:18:21.594568 1916 log.cc:824] T 00000000000000000000000000000000 P 61b8e6f14b0a46f99eb43653e7b9b450: Log is configured to *not* fsync() on all Append() calls
I20260809 02:18:21.596410 1916 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61b8e6f14b0a46f99eb43653e7b9b450: No bootstrap required, opened a new log
I20260809 02:18:21.600766 1916 raft_consensus.cc:348] T 00000000000000000000000000000000 P 61b8e6f14b0a46f99eb43653e7b9b450 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61b8e6f14b0a46f99eb43653e7b9b450" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 44007 } }
Aug 09 02:18:21 dist-test-slave-s11t krb5kdc[1124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241901, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:18:21.601037 1916 raft_consensus.cc:374] T 00000000000000000000000000000000 P 61b8e6f14b0a46f99eb43653e7b9b450 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:18:21.601140 1916 raft_consensus.cc:729] T 00000000000000000000000000000000 P 61b8e6f14b0a46f99eb43653e7b9b450 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61b8e6f14b0a46f99eb43653e7b9b450, State: Initialized, Role: FOLLOWER
I20260809 02:18:21.601650 1916 consensus_queue.cc:260] T 00000000000000000000000000000000 P 61b8e6f14b0a46f99eb43653e7b9b450 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61b8e6f14b0a46f99eb43653e7b9b450" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 44007 } }
I20260809 02:18:21.601857 1916 raft_consensus.cc:388] T 00000000000000000000000000000000 P 61b8e6f14b0a46f99eb43653e7b9b450 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:18:21.601949 1916 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61b8e6f14b0a46f99eb43653e7b9b450 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:18:21.602075 1916 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 61b8e6f14b0a46f99eb43653e7b9b450 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:18:21.603257 1916 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61b8e6f14b0a46f99eb43653e7b9b450 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61b8e6f14b0a46f99eb43653e7b9b450" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 44007 } }
I20260809 02:18:21.603684 1916 leader_election.cc:302] T 00000000000000000000000000000000 P 61b8e6f14b0a46f99eb43653e7b9b450 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 61b8e6f14b0a46f99eb43653e7b9b450; no voters:
I20260809 02:18:21.604075 1916 leader_election.cc:288] T 00000000000000000000000000000000 P 61b8e6f14b0a46f99eb43653e7b9b450 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:18:21.605460 1916 sys_catalog.cc:564] T 00000000000000000000000000000000 P 61b8e6f14b0a46f99eb43653e7b9b450 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:18:21.607813 1916 ranger_client.cc:318] Using new properties file: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 02:18:21.608793 1921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:21.591782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48170 (local address 127.15.75.190:44007)
0809 02:18:21.592256 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:21.592283 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:21.592328 (+ 45us) server_negotiation.cc:407] Connection header received
0809 02:18:21.593182 (+ 854us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:21.593206 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:21.593630 (+ 424us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:21.593971 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:21.594878 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:21.596391 (+ 1513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:21.597670 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:21.598371 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:21.598526 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:21.601891 (+ 3365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:21.601911 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:21.601928 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:18:21.601994 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:21.604940 (+ 2946us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:21.605465 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:21.605472 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:21.605481 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:21.605611 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:21.605959 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:21.605962 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:21.605964 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:21.606499 (+ 535us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:21.607451 (+ 952us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:21.607605 (+ 154us) server_negotiation.cc:299] Negotiation successful
0809 02:18:21.607882 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":130,"threads_started":1}
I20260809 02:18:21.608996 1929 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 61b8e6f14b0a46f99eb43653e7b9b450 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:18:21.611423 1929 raft_consensus.cc:686] T 00000000000000000000000000000000 P 61b8e6f14b0a46f99eb43653e7b9b450 [term 1 LEADER]: Becoming Leader. State: Replica: 61b8e6f14b0a46f99eb43653e7b9b450, State: Running, Role: LEADER
I20260809 02:18:21.612514 1929 consensus_queue.cc:237] T 00000000000000000000000000000000 P 61b8e6f14b0a46f99eb43653e7b9b450 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61b8e6f14b0a46f99eb43653e7b9b450" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 44007 } }
I20260809 02:18:21.615171 1928 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61b8e6f14b0a46f99eb43653e7b9b450 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "61b8e6f14b0a46f99eb43653e7b9b450" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61b8e6f14b0a46f99eb43653e7b9b450" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 44007 } } }
I20260809 02:18:21.615221 1947 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61b8e6f14b0a46f99eb43653e7b9b450 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 61b8e6f14b0a46f99eb43653e7b9b450. Latest consensus state: current_term: 1 leader_uuid: "61b8e6f14b0a46f99eb43653e7b9b450" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61b8e6f14b0a46f99eb43653e7b9b450" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 44007 } } }
I20260809 02:18:21.615365 1928 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61b8e6f14b0a46f99eb43653e7b9b450 [sys.catalog]: This master's current role is: LEADER
I20260809 02:18:21.615365 1947 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61b8e6f14b0a46f99eb43653e7b9b450 [sys.catalog]: This master's current role is: LEADER
I20260809 02:18:21.615965 1950 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:18:21.620966 1950 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:18:21.628100 1950 catalog_manager.cc:1347] Generated new cluster ID: 4477a523b680457fac94cf0e7367e087
I20260809 02:18:21.628160 1950 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:18:21.648840 1950 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:18:21.650276 1950 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:18:21.660871 1950 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 61b8e6f14b0a46f99eb43653e7b9b450: Generated new TSK 0
I20260809 02:18:21.661789 1950 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.75.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab.
Entry for principal kudu/127.15.75.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab.
I20260809 02:18:22.343904 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.75.148:44415/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.129:0
--local_ip_for_outbound_sockets=127.15.75.129
--webserver_interface=127.15.75.129
--webserver_port=0
--tserver_master_addrs=127.15.75.190:44007
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab
--principal=kudu/127.15.75.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:36263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:18:22.507992 1995 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:18:22.508378 1995 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:18:22.508472 1995 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:18:22.514963 1995 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:18:22.515033 1995 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:18:22.515134 1995 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.129
I20260809 02:18:22.522990 1995 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:36263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.15.75.148:44415/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab
--principal=kudu/127.15.75.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.75.129
--webserver_port=0
--tserver_master_addrs=127.15.75.190:44007
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1995
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.129
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:18:22.524410 1995 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:18:22.525810 1995 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:18:22.539212 2000 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:22.539216 2004 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:22.539220 2006 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:22.539866 1995 server_base.cc:1034] running on GCE node
I20260809 02:18:22.540270 1995 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:18:22.541028 1995 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:18:22.548467 1995 hybrid_clock.cc:648] HybridClock initialized: now 1786241902548412 us; error 63 us; skew 500 ppm
Aug 09 02:18:22 dist-test-slave-s11t krb5kdc[1124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241902, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:18:22.551817 1995 init.cc:377] Logged in from keytab as kudu/127.15.75.129@KRBTEST.COM (short username kudu)
I20260809 02:18:22.552835 1995 webserver.cc:460] Webserver started at http://127.15.75.129:36729/ using document root <none> and password file <none>
I20260809 02:18:22.553416 1995 fs_manager.cc:359] Metadata directory not provided
I20260809 02:18:22.553546 1995 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:18:22.553792 1995 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 02:18:22 dist-test-slave-s11t krb5kdc[1124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241902, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.129@KRBTEST.COM for HTTP/127.15.75.148@KRBTEST.COM
I20260809 02:18:22.570864 1995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-0/data/instance:
uuid: "f4a558685b5245aea197dacba4d07b07"
format_stamp: "Formatted at 2026-08-09 02:18:22 on dist-test-slave-s11t"
server_key: "45a0d6644bcf8214190a0937d7bba602"
server_key_iv: "f3d95f527914487b45418f1f789e747d"
server_key_version: "kuduclusterkey@0"
I20260809 02:18:22.571486 1995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-0/wal/instance:
uuid: "f4a558685b5245aea197dacba4d07b07"
format_stamp: "Formatted at 2026-08-09 02:18:22 on dist-test-slave-s11t"
server_key: "45a0d6644bcf8214190a0937d7bba602"
server_key_iv: "f3d95f527914487b45418f1f789e747d"
server_key_version: "kuduclusterkey@0"
I20260809 02:18:22.575919 1995 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260809 02:18:22.604023 2020 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:22.605845 1995 fs_manager.cc:725] Time spent opening block manager: real 0.012s user 0.002s sys 0.000s
I20260809 02:18:22.606081 1995 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-0/wal
uuid: "f4a558685b5245aea197dacba4d07b07"
format_stamp: "Formatted at 2026-08-09 02:18:22 on dist-test-slave-s11t"
server_key: "45a0d6644bcf8214190a0937d7bba602"
server_key_iv: "f3d95f527914487b45418f1f789e747d"
server_key_version: "kuduclusterkey@0"
I20260809 02:18:22.606338 1995 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:18:22.635507 1995 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:18:22.639253 1995 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:18:22.639510 1995 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:18:22.640249 1995 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:18:22.641397 1995 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:18:22.641515 1995 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:22.641644 1995 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:18:22.642511 1995 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260809 02:18:22.653390 1995 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.129:38959
I20260809 02:18:22.654590 1995 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
I20260809 02:18:22.655234 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 1995
I20260809 02:18:22.655364 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-0/wal/instance
I20260809 02:18:22.655700 15662 external_mini_cluster.cc:1428] Setting key 6f8afc4e61e5a83e3320231dfd918c28
I20260809 02:18:22.658023 2222 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.129:38959 every 8 connection(s)
WARNING: no policy specified for kudu/127.15.75.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.130@KRBTEST.COM" created.
Aug 09 02:18:22 dist-test-slave-s11t krb5kdc[1124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241902, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.129@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:18:22.685293 2230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:22.660307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:44007 (local address 127.15.75.129:54761)
0809 02:18:22.662148 (+ 1841us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:22.662206 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:22.663236 (+ 1030us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:22.667668 (+ 4432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:22.667677 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:22.668047 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:18:22.668733 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:22.668756 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:22.672621 (+ 3865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:22.672624 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:22.674083 (+ 1459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:22.674095 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:22.674799 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:22.674804 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:22.675013 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:22.675732 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:18:22.675753 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:18:22.679358 (+ 3605us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:18:22.681774 (+ 2416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:18:22.681780 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:18:22.681797 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:18:22.682167 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:18:22.682542 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:18:22.682545 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:18:22.682547 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:18:22.682765 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:18:22.683260 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:18:22.683267 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:18:22.683604 (+ 337us) client_negotiation.cc:769] Sending connection context
0809 02:18:22.683821 (+ 217us) client_negotiation.cc:240] Negotiation successful
0809 02:18:22.684153 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1612,"thread_start_us":162,"threads_started":1}
I20260809 02:18:22.686262 2234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:22.660900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.129:54761 (local address 127.15.75.190:44007)
0809 02:18:22.667245 (+ 6345us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:22.667254 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:22.667270 (+ 16us) server_negotiation.cc:407] Connection header received
0809 02:18:22.667319 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:22.667322 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:22.667374 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:22.667487 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:22.671545 (+ 4058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:22.672447 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:22.674261 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:22.674617 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:22.674724 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:22.679578 (+ 4854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:22.679582 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:22.679585 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:18:22.679618 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:22.681632 (+ 2014us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:22.682316 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:22.682320 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:22.682322 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:22.682421 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:22.682899 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:22.682904 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:22.682908 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:22.683134 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:22.685852 (+ 2718us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:22.685965 (+ 113us) server_negotiation.cc:299] Negotiation successful
0809 02:18:22.686097 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6259,"thread_start_us":116,"threads_started":1}
I20260809 02:18:22.687500 2224 heartbeater.cc:347] Connected to a master server at 127.15.75.190:44007
I20260809 02:18:22.687775 2224 heartbeater.cc:464] Registering TS with master...
I20260809 02:18:22.688346 2224 heartbeater.cc:511] Master 127.15.75.190:44007 requested a full tablet report, sending...
I20260809 02:18:22.690093 1865 ts_manager.cc:194] Registered new tserver with Master: f4a558685b5245aea197dacba4d07b07 (127.15.75.129:38959)
I20260809 02:18:22.691597 1865 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.75.129@KRBTEST.COM'} at 127.15.75.129:54761
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab.
Entry for principal kudu/127.15.75.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab.
I20260809 02:18:22.698254 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.75.148:44415/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.130:0
--local_ip_for_outbound_sockets=127.15.75.130
--webserver_interface=127.15.75.130
--webserver_port=0
--tserver_master_addrs=127.15.75.190:44007
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab
--principal=kudu/127.15.75.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:36263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 02:18:22 dist-test-slave-s11t krb5kdc[1124](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786241902, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260809 02:18:22.857815 2237 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:18:22.858232 2237 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:18:22.858342 2237 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:18:22.864323 2237 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:18:22.864463 2237 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:18:22.864600 2237 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.130
I20260809 02:18:22.872313 2237 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:36263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.15.75.148:44415/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab
--principal=kudu/127.15.75.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.75.130
--webserver_port=0
--tserver_master_addrs=127.15.75.190:44007
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2237
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.130
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:18:22.873673 2237 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:18:22.875100 2237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:18:22.887135 2251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:22.887117 2256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:22.887612 2252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:22.887739 2237 server_base.cc:1034] running on GCE node
I20260809 02:18:22.888355 2237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:18:22.889173 2237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:18:22.890395 2237 hybrid_clock.cc:648] HybridClock initialized: now 1786241902890377 us; error 31 us; skew 500 ppm
Aug 09 02:18:22 dist-test-slave-s11t krb5kdc[1124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241902, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:18:22.894192 2237 init.cc:377] Logged in from keytab as kudu/127.15.75.130@KRBTEST.COM (short username kudu)
I20260809 02:18:22.895319 2237 webserver.cc:460] Webserver started at http://127.15.75.130:33985/ using document root <none> and password file <none>
I20260809 02:18:22.896006 2237 fs_manager.cc:359] Metadata directory not provided
I20260809 02:18:22.896095 2237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:18:22.896351 2237 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 02:18:22 dist-test-slave-s11t krb5kdc[1124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241902, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.130@KRBTEST.COM for HTTP/127.15.75.148@KRBTEST.COM
I20260809 02:18:22.911206 2237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-1/data/instance:
uuid: "37e0dba10c8944de86fbe4f8c414c43c"
format_stamp: "Formatted at 2026-08-09 02:18:22 on dist-test-slave-s11t"
server_key: "e2e04ee9543637d067a327171ccab4c6"
server_key_iv: "0425d981c45a6633f6069479357c1e77"
server_key_version: "kuduclusterkey@0"
I20260809 02:18:22.912087 2237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-1/wal/instance:
uuid: "37e0dba10c8944de86fbe4f8c414c43c"
format_stamp: "Formatted at 2026-08-09 02:18:22 on dist-test-slave-s11t"
server_key: "e2e04ee9543637d067a327171ccab4c6"
server_key_iv: "0425d981c45a6633f6069479357c1e77"
server_key_version: "kuduclusterkey@0"
I20260809 02:18:22.917178 2237 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 02:18:22.936441 2264 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:22.938308 2237 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 02:18:22.938445 2237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-1/wal
uuid: "37e0dba10c8944de86fbe4f8c414c43c"
format_stamp: "Formatted at 2026-08-09 02:18:22 on dist-test-slave-s11t"
server_key: "e2e04ee9543637d067a327171ccab4c6"
server_key_iv: "0425d981c45a6633f6069479357c1e77"
server_key_version: "kuduclusterkey@0"
I20260809 02:18:22.938652 2237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:18:22.971761 2237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:18:22.975306 2237 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:18:22.975548 2237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:18:22.976466 2237 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:18:22.977819 2237 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:18:22.977897 2237 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:22.977962 2237 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:18:22.977998 2237 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:22.990520 2237 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.130:38111
I20260809 02:18:22.990548 2454 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.130:38111 every 8 connection(s)
I20260809 02:18:22.991536 2237 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
I20260809 02:18:22.998296 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 2237
I20260809 02:18:22.998432 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-1/wal/instance
I20260809 02:18:22.998764 15662 external_mini_cluster.cc:1428] Setting key c8ca64c37e1c1dfa4d890d3d36e09eec
Aug 09 02:18:22 dist-test-slave-s11t krb5kdc[1124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241902, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.130@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:18:23.008107 2234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:22.993547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.130:52799 (local address 127.15.75.190:44007)
0809 02:18:22.993683 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:22.993687 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:22.994620 (+ 933us) server_negotiation.cc:407] Connection header received
0809 02:18:22.995610 (+ 990us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:22.995613 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:22.995674 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:22.995767 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:22.997195 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:22.997749 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:22.999379 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:22.999776 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:22.999882 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:23.003224 (+ 3342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:23.003228 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:23.003231 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:18:23.003260 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:23.005106 (+ 1846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:23.006031 (+ 925us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:23.006035 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:23.006037 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:23.006125 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:23.006668 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:23.006672 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:23.006674 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:23.006927 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:23.007003 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:23.007797 (+ 794us) server_negotiation.cc:299] Negotiation successful
0809 02:18:23.007931 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":54}
I20260809 02:18:23.010017 2458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:22.993986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:44007 (local address 127.15.75.130:52799)
0809 02:18:22.994466 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:22.994518 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:22.995394 (+ 876us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:22.996080 (+ 686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:22.996089 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:22.996438 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:18:22.997040 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:22.997054 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:22.997885 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:22.997888 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:22.999233 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:22.999244 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:23.000056 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:23.000060 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:23.000501 (+ 441us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:23.001218 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:18:23.001241 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:18:23.003009 (+ 1768us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:18:23.005542 (+ 2533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:18:23.005548 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:18:23.005566 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:18:23.005906 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:18:23.006254 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:18:23.006257 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:18:23.006259 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:18:23.006556 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:18:23.007115 (+ 559us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:18:23.007122 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:18:23.007505 (+ 383us) client_negotiation.cc:769] Sending connection context
0809 02:18:23.008634 (+ 1129us) client_negotiation.cc:240] Negotiation successful
0809 02:18:23.009005 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":156,"threads_started":1}
I20260809 02:18:23.015232 2456 heartbeater.cc:347] Connected to a master server at 127.15.75.190:44007
I20260809 02:18:23.015518 2456 heartbeater.cc:464] Registering TS with master...
I20260809 02:18:23.016096 2456 heartbeater.cc:511] Master 127.15.75.190:44007 requested a full tablet report, sending...
I20260809 02:18:23.017283 1865 ts_manager.cc:194] Registered new tserver with Master: 37e0dba10c8944de86fbe4f8c414c43c (127.15.75.130:38111)
WARNING: no policy specified for kudu/127.15.75.131@KRBTEST.COM; defaulting to no policy
I20260809 02:18:23.018064 1865 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.75.130@KRBTEST.COM'} at 127.15.75.130:52799
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab.
Entry for principal kudu/127.15.75.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab.
I20260809 02:18:23.039882 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.75.148:44415/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.131:0
--local_ip_for_outbound_sockets=127.15.75.131
--webserver_interface=127.15.75.131
--webserver_port=0
--tserver_master_addrs=127.15.75.190:44007
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab
--principal=kudu/127.15.75.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:36263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 02:18:23 dist-test-slave-s11t krb5kdc[1124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241902, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.190@KRBTEST.COM for HTTP/127.15.75.148@KRBTEST.COM
W20260809 02:18:23.199972 2462 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:18:23.200429 2462 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:18:23.200600 2462 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:18:23.206452 2462 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:18:23.206583 2462 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:18:23.206724 2462 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.131
I20260809 02:18:23.215158 2462 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:36263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.15.75.148:44415/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab
--principal=kudu/127.15.75.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.75.131
--webserver_port=0
--tserver_master_addrs=127.15.75.190:44007
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2462
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.131
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:18:23.216756 2462 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:18:23.218869 2462 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:18:23.232156 2471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:23.232169 2474 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:23.232757 2470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:23.234062 2462 server_base.cc:1034] running on GCE node
I20260809 02:18:23.234508 2462 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:18:23.235276 2462 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:18:23.238593 2462 hybrid_clock.cc:648] HybridClock initialized: now 1786241903238524 us; error 98 us; skew 500 ppm
Aug 09 02:18:23 dist-test-slave-s11t krb5kdc[1124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241903, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:18:23.242437 2462 init.cc:377] Logged in from keytab as kudu/127.15.75.131@KRBTEST.COM (short username kudu)
I20260809 02:18:23.243657 2462 webserver.cc:460] Webserver started at http://127.15.75.131:38621/ using document root <none> and password file <none>
I20260809 02:18:23.244585 2462 fs_manager.cc:359] Metadata directory not provided
I20260809 02:18:23.244745 2462 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:18:23.245047 2462 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 02:18:23 dist-test-slave-s11t krb5kdc[1124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241903, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.131@KRBTEST.COM for HTTP/127.15.75.148@KRBTEST.COM
I20260809 02:18:23.261062 2462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-2/data/instance:
uuid: "b68bbc0c4f0c47479e75790e960df666"
format_stamp: "Formatted at 2026-08-09 02:18:23 on dist-test-slave-s11t"
server_key: "af445fbb9587283a73b080d137b37732"
server_key_iv: "1ee9bd1d6ba13a7fb90c1af6288a4451"
server_key_version: "kuduclusterkey@0"
I20260809 02:18:23.261989 2462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-2/wal/instance:
uuid: "b68bbc0c4f0c47479e75790e960df666"
format_stamp: "Formatted at 2026-08-09 02:18:23 on dist-test-slave-s11t"
server_key: "af445fbb9587283a73b080d137b37732"
server_key_iv: "1ee9bd1d6ba13a7fb90c1af6288a4451"
server_key_version: "kuduclusterkey@0"
I20260809 02:18:23.267273 2462 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 02:18:23.282979 2483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:23.284458 2462 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 02:18:23.284606 2462 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-2/wal
uuid: "b68bbc0c4f0c47479e75790e960df666"
format_stamp: "Formatted at 2026-08-09 02:18:23 on dist-test-slave-s11t"
server_key: "af445fbb9587283a73b080d137b37732"
server_key_iv: "1ee9bd1d6ba13a7fb90c1af6288a4451"
server_key_version: "kuduclusterkey@0"
I20260809 02:18:23.284807 2462 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:18:23.325470 2462 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:18:23.329186 2462 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:18:23.329384 2462 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:18:23.330121 2462 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:18:23.331135 2462 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:18:23.331195 2462 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:23.331245 2462 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:18:23.331321 2462 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260809 02:18:23.342161 2462 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.131:44853
I20260809 02:18:23.342242 2667 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.131:44853 every 8 connection(s)
I20260809 02:18:23.343349 2462 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
I20260809 02:18:23.349660 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 2462
I20260809 02:18:23.349802 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-2/wal/instance
I20260809 02:18:23.350118 15662 external_mini_cluster.cc:1428] Setting key 856e7591bfad0210599aaafb1d995d18
Aug 09 02:18:23 dist-test-slave-s11t krb5kdc[1124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241903, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.131@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:18:23.361145 2234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:23.345630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.131:41531 (local address 127.15.75.190:44007)
0809 02:18:23.345776 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:23.345781 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:23.346875 (+ 1094us) server_negotiation.cc:407] Connection header received
0809 02:18:23.348273 (+ 1398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:23.348276 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:23.348335 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:23.348431 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:23.350131 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:23.351023 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:23.353067 (+ 2044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:23.353432 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:23.353519 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:23.356773 (+ 3254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:23.356777 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:23.356781 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:18:23.356808 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:23.358576 (+ 1768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:23.359216 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:23.359220 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:23.359222 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:23.359313 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:23.359761 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:23.359764 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:23.359766 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:23.359998 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:23.360089 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:23.360834 (+ 745us) server_negotiation.cc:299] Negotiation successful
0809 02:18:23.360977 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 02:18:23.362946 2674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:23.346101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:44007 (local address 127.15.75.131:41531)
0809 02:18:23.346692 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:23.346751 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:23.347952 (+ 1201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:23.348644 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:23.348656 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:23.349061 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:18:23.349944 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:23.349967 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:23.351459 (+ 1492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:23.351462 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:23.352920 (+ 1458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:23.352931 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:23.353733 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:23.353738 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:23.353954 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:23.354688 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:18:23.354711 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:18:23.356544 (+ 1833us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:18:23.358715 (+ 2171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:18:23.358721 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:18:23.358737 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:18:23.359093 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:18:23.359451 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:18:23.359454 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:18:23.359456 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:18:23.359635 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:18:23.360161 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:18:23.360168 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:18:23.360502 (+ 334us) client_negotiation.cc:769] Sending connection context
0809 02:18:23.361441 (+ 939us) client_negotiation.cc:240] Negotiation successful
0809 02:18:23.361792 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":164,"threads_started":1}
I20260809 02:18:23.364589 2670 heartbeater.cc:347] Connected to a master server at 127.15.75.190:44007
I20260809 02:18:23.364858 2670 heartbeater.cc:464] Registering TS with master...
I20260809 02:18:23.365482 2670 heartbeater.cc:511] Master 127.15.75.190:44007 requested a full tablet report, sending...
I20260809 02:18:23.366631 1864 ts_manager.cc:194] Registered new tserver with Master: b68bbc0c4f0c47479e75790e960df666 (127.15.75.131:44853)
I20260809 02:18:23.367480 1864 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.75.131@KRBTEST.COM'} at 127.15.75.131:41531
I20260809 02:18:23.376963 15662 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 02:18:23.377746 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 1995
I20260809 02:18:23.394037 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 2237
I20260809 02:18:23.402905 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 2462
I20260809 02:18:23.413518 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 1820
I20260809 02:18:23.427654 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:44007
--webserver_interface=127.15.75.190
--webserver_port=32865
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab
--principal=kudu/127.15.75.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:36263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.75.190:44007
--ranger_config_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.75.148:44415/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.15.75.148:44415/kms with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:18:23.538005 2675 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:18:23.538236 2675 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:18:23.538273 2675 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:18:23.542318 2675 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:18:23.542368 2675 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:18:23.542388 2675 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:18:23.542404 2675 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:18:23.542419 2675 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:18:23.547616 2675 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:36263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.15.75.148:44415/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.15.75.190:44007
--ranger_config_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab
--principal=kudu/127.15.75.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:44007
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.75.190
--webserver_port=32865
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2675
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:18:23.548486 2675 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:18:23.549460 2675 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:18:23.557684 2683 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:23.557713 2688 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:23.557662 2684 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:23.557927 2675 server_base.cc:1034] running on GCE node
I20260809 02:18:23.558559 2675 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:18:23.559659 2675 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:18:23.560839 2675 hybrid_clock.cc:648] HybridClock initialized: now 1786241903560814 us; error 43 us; skew 500 ppm
Aug 09 02:18:23 dist-test-slave-s11t krb5kdc[1124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241903, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:18:23.563467 2675 init.cc:377] Logged in from keytab as kudu/127.15.75.190@KRBTEST.COM (short username kudu)
I20260809 02:18:23.564337 2675 webserver.cc:460] Webserver started at http://127.15.75.190:32865/ using document root <none> and password file <none>
I20260809 02:18:23.564929 2675 fs_manager.cc:359] Metadata directory not provided
I20260809 02:18:23.565030 2675 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 02:18:23 dist-test-slave-s11t krb5kdc[1124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241903, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.190@KRBTEST.COM for HTTP/127.15.75.148@KRBTEST.COM
I20260809 02:18:23.643412 2675 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 02:18:23.645736 2698 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:23.647074 2675 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 02:18:23.647209 2675 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/master-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/master-0/wal
uuid: "61b8e6f14b0a46f99eb43653e7b9b450"
format_stamp: "Formatted at 2026-08-09 02:18:21 on dist-test-slave-s11t"
server_key: "706c37f8b5bb96f986565214f34e970f"
server_key_iv: "6cbd2ddec6f7ebb22972dbb774c6a5dc"
server_key_version: "kuduclusterkey@0"
I20260809 02:18:23.647753 2675 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:18:23.658012 2675 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:18:23.660722 2675 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:18:23.660938 2675 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:18:23.668179 2675 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.190:44007
I20260809 02:18:23.668205 2777 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.190:44007 every 8 connection(s)
I20260809 02:18:23.669291 2675 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
I20260809 02:18:23.673409 2779 sys_catalog.cc:263] Verifying existing consensus state
I20260809 02:18:23.675079 2779 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61b8e6f14b0a46f99eb43653e7b9b450: Bootstrap starting.
I20260809 02:18:23.675354 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 2675
I20260809 02:18:23.675814 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.75.148:44415/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.129:38959
--local_ip_for_outbound_sockets=127.15.75.129
--tserver_master_addrs=127.15.75.190:44007
--webserver_port=36729
--webserver_interface=127.15.75.129
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab
--principal=kudu/127.15.75.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:36263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.15.75.148:44415/kms with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 02:18:23.691756 2779 log.cc:824] T 00000000000000000000000000000000 P 61b8e6f14b0a46f99eb43653e7b9b450: Log is configured to *not* fsync() on all Append() calls
I20260809 02:18:23.698243 2779 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61b8e6f14b0a46f99eb43653e7b9b450: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 02:18:23.698643 2779 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61b8e6f14b0a46f99eb43653e7b9b450: Bootstrap complete.
I20260809 02:18:23.702767 2779 raft_consensus.cc:348] T 00000000000000000000000000000000 P 61b8e6f14b0a46f99eb43653e7b9b450 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61b8e6f14b0a46f99eb43653e7b9b450" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 44007 } }
I20260809 02:18:23.703168 2779 raft_consensus.cc:729] T 00000000000000000000000000000000 P 61b8e6f14b0a46f99eb43653e7b9b450 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61b8e6f14b0a46f99eb43653e7b9b450, State: Initialized, Role: FOLLOWER
I20260809 02:18:23.703694 2779 consensus_queue.cc:260] T 00000000000000000000000000000000 P 61b8e6f14b0a46f99eb43653e7b9b450 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61b8e6f14b0a46f99eb43653e7b9b450" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 44007 } }
I20260809 02:18:23.703840 2779 raft_consensus.cc:388] T 00000000000000000000000000000000 P 61b8e6f14b0a46f99eb43653e7b9b450 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:18:23.703928 2779 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61b8e6f14b0a46f99eb43653e7b9b450 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:18:23.704064 2779 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 61b8e6f14b0a46f99eb43653e7b9b450 [term 1 FOLLOWER]: Advancing to term 2
I20260809 02:18:23.705957 2779 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61b8e6f14b0a46f99eb43653e7b9b450 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61b8e6f14b0a46f99eb43653e7b9b450" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 44007 } }
I20260809 02:18:23.706287 2779 leader_election.cc:302] T 00000000000000000000000000000000 P 61b8e6f14b0a46f99eb43653e7b9b450 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 61b8e6f14b0a46f99eb43653e7b9b450; no voters:
I20260809 02:18:23.706641 2779 leader_election.cc:288] T 00000000000000000000000000000000 P 61b8e6f14b0a46f99eb43653e7b9b450 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 02:18:23.706724 2789 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 61b8e6f14b0a46f99eb43653e7b9b450 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 02:18:23.706977 2789 raft_consensus.cc:686] T 00000000000000000000000000000000 P 61b8e6f14b0a46f99eb43653e7b9b450 [term 2 LEADER]: Becoming Leader. State: Replica: 61b8e6f14b0a46f99eb43653e7b9b450, State: Running, Role: LEADER
I20260809 02:18:23.707317 2789 consensus_queue.cc:237] T 00000000000000000000000000000000 P 61b8e6f14b0a46f99eb43653e7b9b450 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61b8e6f14b0a46f99eb43653e7b9b450" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 44007 } }
I20260809 02:18:23.707880 2779 sys_catalog.cc:564] T 00000000000000000000000000000000 P 61b8e6f14b0a46f99eb43653e7b9b450 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:18:23.708705 2790 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61b8e6f14b0a46f99eb43653e7b9b450 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "61b8e6f14b0a46f99eb43653e7b9b450" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61b8e6f14b0a46f99eb43653e7b9b450" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 44007 } } }
I20260809 02:18:23.708837 2790 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61b8e6f14b0a46f99eb43653e7b9b450 [sys.catalog]: This master's current role is: LEADER
I20260809 02:18:23.709206 2791 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61b8e6f14b0a46f99eb43653e7b9b450 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 61b8e6f14b0a46f99eb43653e7b9b450. Latest consensus state: current_term: 2 leader_uuid: "61b8e6f14b0a46f99eb43653e7b9b450" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61b8e6f14b0a46f99eb43653e7b9b450" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 44007 } } }
I20260809 02:18:23.709344 2791 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61b8e6f14b0a46f99eb43653e7b9b450 [sys.catalog]: This master's current role is: LEADER
I20260809 02:18:23.709656 2806 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:18:23.712903 2806 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:18:23.713415 2779 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 02:18:23.713634 2806 catalog_manager.cc:1259] Loaded cluster ID: 4477a523b680457fac94cf0e7367e087
I20260809 02:18:23.713734 2806 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:18:23.716032 2806 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:18:23.716713 2806 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 61b8e6f14b0a46f99eb43653e7b9b450: Loaded TSK: 0
I20260809 02:18:23.717917 2806 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 02:18:23.795709 2785 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:18:23.796149 2785 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:18:23.796288 2785 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:18:23.801483 2785 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:18:23.801563 2785 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:18:23.801684 2785 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.129
I20260809 02:18:23.807148 2785 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:36263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.15.75.148:44415/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab
--principal=kudu/127.15.75.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.129:38959
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.75.129
--webserver_port=36729
--tserver_master_addrs=127.15.75.190:44007
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2785
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.129
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:18:23.808377 2785 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:18:23.809696 2785 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:18:23.820756 2844 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:23.820842 2842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:23.821169 2847 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:23.821274 2785 server_base.cc:1034] running on GCE node
I20260809 02:18:23.821733 2785 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:18:23.822445 2785 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:18:23.823634 2785 hybrid_clock.cc:648] HybridClock initialized: now 1786241903823598 us; error 46 us; skew 500 ppm
Aug 09 02:18:23 dist-test-slave-s11t krb5kdc[1124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241903, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:18:23.827044 2785 init.cc:377] Logged in from keytab as kudu/127.15.75.129@KRBTEST.COM (short username kudu)
I20260809 02:18:23.828121 2785 webserver.cc:460] Webserver started at http://127.15.75.129:36729/ using document root <none> and password file <none>
I20260809 02:18:23.828886 2785 fs_manager.cc:359] Metadata directory not provided
I20260809 02:18:23.829020 2785 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 02:18:23 dist-test-slave-s11t krb5kdc[1124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241903, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.129@KRBTEST.COM for HTTP/127.15.75.148@KRBTEST.COM
I20260809 02:18:23.854313 2785 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.002s sys 0.001s
I20260809 02:18:23.868122 2858 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:23.870142 2785 fs_manager.cc:725] Time spent opening block manager: real 0.014s user 0.002s sys 0.000s
I20260809 02:18:23.870313 2785 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-0/wal
uuid: "f4a558685b5245aea197dacba4d07b07"
format_stamp: "Formatted at 2026-08-09 02:18:22 on dist-test-slave-s11t"
server_key: "45a0d6644bcf8214190a0937d7bba602"
server_key_iv: "f3d95f527914487b45418f1f789e747d"
server_key_version: "kuduclusterkey@0"
I20260809 02:18:23.870930 2785 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:18:23.894503 2785 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:18:23.904857 2785 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:18:23.905108 2785 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:18:23.905969 2785 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:18:23.907137 2785 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:18:23.907209 2785 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:23.907392 2785 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:18:23.907435 2785 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:23.922181 2785 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.129:38959
I20260809 02:18:23.927493 3050 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.129:38959 every 8 connection(s)
I20260809 02:18:23.928022 2785 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
I20260809 02:18:23.934132 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 2785
I20260809 02:18:23.934675 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.75.148:44415/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.130:38111
--local_ip_for_outbound_sockets=127.15.75.130
--tserver_master_addrs=127.15.75.190:44007
--webserver_port=33985
--webserver_interface=127.15.75.130
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab
--principal=kudu/127.15.75.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:36263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.15.75.148:44415/kms with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 02:18:23 dist-test-slave-s11t krb5kdc[1124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241903, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.129@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:18:23.961270 3056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:23.937171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:44007 (local address 127.15.75.129:36949)
0809 02:18:23.940019 (+ 2848us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:23.940078 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:23.941181 (+ 1103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:23.944821 (+ 3640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:23.944840 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:23.945246 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:18:23.945993 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:23.946017 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:23.948123 (+ 2106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:23.948127 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:23.949687 (+ 1560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:23.949700 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:23.951101 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:23.951107 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:23.951416 (+ 309us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:23.952237 (+ 821us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:18:23.952259 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:18:23.954293 (+ 2034us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:18:23.957374 (+ 3081us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:18:23.957381 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:18:23.957396 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:18:23.957770 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:18:23.958233 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:18:23.958236 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:18:23.958238 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:18:23.958449 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:18:23.959298 (+ 849us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:18:23.959305 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:18:23.959603 (+ 298us) client_negotiation.cc:769] Sending connection context
0809 02:18:23.959802 (+ 199us) client_negotiation.cc:240] Negotiation successful
0809 02:18:23.960149 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2588,"thread_start_us":163,"threads_started":1}
I20260809 02:18:23.962957 3057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:23.938715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.129:36949 (local address 127.15.75.190:44007)
0809 02:18:23.942942 (+ 4227us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:23.942967 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:23.943016 (+ 49us) server_negotiation.cc:407] Connection header received
0809 02:18:23.943332 (+ 316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:23.943350 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:23.943803 (+ 453us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:23.944218 (+ 415us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:23.946218 (+ 2000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:23.947950 (+ 1732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:23.949856 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:23.950689 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:23.950864 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:23.954769 (+ 3905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:23.954776 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:23.954794 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:18:23.954830 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:23.957200 (+ 2370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:23.957946 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:23.957952 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:23.957958 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:23.958107 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:23.958582 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:23.958586 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:23.958588 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:23.959099 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:23.959238 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:23.961666 (+ 2428us) server_negotiation.cc:299] Negotiation successful
0809 02:18:23.961977 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4050,"thread_start_us":123,"threads_started":1}
I20260809 02:18:23.966220 3053 heartbeater.cc:347] Connected to a master server at 127.15.75.190:44007
I20260809 02:18:23.966475 3053 heartbeater.cc:464] Registering TS with master...
I20260809 02:18:23.967087 3053 heartbeater.cc:511] Master 127.15.75.190:44007 requested a full tablet report, sending...
W20260809 02:18:23.969514 3053 heartbeater.cc:644] Failed to heartbeat to 127.15.75.190:44007 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 02:18:23.969651 3053 heartbeater.cc:464] Registering TS with master...
I20260809 02:18:23.969849 3053 heartbeater.cc:511] Master 127.15.75.190:44007 requested a full tablet report, sending...
I20260809 02:18:23.970788 3053 heartbeater.cc:464] Registering TS with master...
I20260809 02:18:23.970966 3053 heartbeater.cc:511] Master 127.15.75.190:44007 requested a full tablet report, sending...
W20260809 02:18:23.971771 3053 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 02:18:24.119626 3055 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:18:24.120044 3055 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:18:24.120160 3055 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:18:24.126003 3055 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:18:24.126138 3055 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:18:24.126266 3055 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.130
I20260809 02:18:24.133656 3055 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:36263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.15.75.148:44415/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab
--principal=kudu/127.15.75.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.130:38111
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.75.130
--webserver_port=33985
--tserver_master_addrs=127.15.75.190:44007
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3055
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.130
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:18:24.135129 3055 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:18:24.136629 3055 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:18:24.149137 3071 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:24.149401 3065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:24.149698 3055 server_base.cc:1034] running on GCE node
W20260809 02:18:24.149843 3064 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:24.150290 3055 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:18:24.151052 3055 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:18:24.152839 3055 hybrid_clock.cc:648] HybridClock initialized: now 1786241904152804 us; error 49 us; skew 500 ppm
Aug 09 02:18:24 dist-test-slave-s11t krb5kdc[1124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241904, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:18:24.156602 3055 init.cc:377] Logged in from keytab as kudu/127.15.75.130@KRBTEST.COM (short username kudu)
I20260809 02:18:24.157783 3055 webserver.cc:460] Webserver started at http://127.15.75.130:33985/ using document root <none> and password file <none>
I20260809 02:18:24.158576 3055 fs_manager.cc:359] Metadata directory not provided
I20260809 02:18:24.158744 3055 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 02:18:24 dist-test-slave-s11t krb5kdc[1124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241904, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.130@KRBTEST.COM for HTTP/127.15.75.148@KRBTEST.COM
I20260809 02:18:24.188711 3055 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 02:18:24.192924 3085 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:24.194725 3055 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260809 02:18:24.194885 3055 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-1/wal
uuid: "37e0dba10c8944de86fbe4f8c414c43c"
format_stamp: "Formatted at 2026-08-09 02:18:22 on dist-test-slave-s11t"
server_key: "e2e04ee9543637d067a327171ccab4c6"
server_key_iv: "0425d981c45a6633f6069479357c1e77"
server_key_version: "kuduclusterkey@0"
I20260809 02:18:24.195621 3055 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:18:24.223301 3055 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:18:24.227698 3055 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:18:24.227923 3055 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:18:24.228663 3055 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:18:24.233758 3055 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:18:24.235244 3055 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 02:18:24.235347 3055 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:18:24.235373 3055 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:24.246364 3055 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.130:38111
I20260809 02:18:24.247651 3055 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
I20260809 02:18:24.248829 3280 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.130:38111 every 8 connection(s)
I20260809 02:18:24.257342 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 3055
I20260809 02:18:24.257894 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.75.148:44415/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.131:44853
--local_ip_for_outbound_sockets=127.15.75.131
--tserver_master_addrs=127.15.75.190:44007
--webserver_port=38621
--webserver_interface=127.15.75.131
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab
--principal=kudu/127.15.75.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:36263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.15.75.148:44415/kms with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 02:18:24 dist-test-slave-s11t krb5kdc[1124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241904, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.130@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:18:24.272413 3057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:24.253285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.130:40771 (local address 127.15.75.190:44007)
0809 02:18:24.253446 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:24.253451 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:24.255412 (+ 1961us) server_negotiation.cc:407] Connection header received
0809 02:18:24.256660 (+ 1248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:24.256663 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:24.256752 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:24.256815 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:24.258465 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:24.259293 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:24.261015 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:24.261314 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:24.261391 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:24.265384 (+ 3993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:24.265388 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:24.265391 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:18:24.265419 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:24.267332 (+ 1913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:24.268077 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:24.268082 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:24.268083 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:24.268184 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:24.268643 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:24.268648 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:24.268653 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:24.268919 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:24.269014 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:24.272036 (+ 3022us) server_negotiation.cc:299] Negotiation successful
0809 02:18:24.272212 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 02:18:24.277484 3286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:24.251860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:44007 (local address 127.15.75.130:40771)
0809 02:18:24.255235 (+ 3375us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:24.255293 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:24.256388 (+ 1095us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:24.257254 (+ 866us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:24.257263 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:24.257616 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:18:24.258287 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:24.258307 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:24.259462 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:24.259465 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:24.260876 (+ 1411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:24.260887 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:24.261595 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:24.261598 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:24.261781 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:24.263293 (+ 1512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:18:24.263314 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:18:24.265190 (+ 1876us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:18:24.267488 (+ 2298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:18:24.267495 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:18:24.267514 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:18:24.267926 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:18:24.268321 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:18:24.268325 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:18:24.268327 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:18:24.268517 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:18:24.270713 (+ 2196us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:18:24.270720 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:18:24.271070 (+ 350us) client_negotiation.cc:769] Sending connection context
0809 02:18:24.273476 (+ 2406us) client_negotiation.cc:240] Negotiation successful
0809 02:18:24.275211 (+ 1735us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3137,"thread_start_us":173,"threads_started":1}
I20260809 02:18:24.280555 3282 heartbeater.cc:347] Connected to a master server at 127.15.75.190:44007
I20260809 02:18:24.280848 3282 heartbeater.cc:464] Registering TS with master...
I20260809 02:18:24.281471 3282 heartbeater.cc:511] Master 127.15.75.190:44007 requested a full tablet report, sending...
W20260809 02:18:24.283315 3282 heartbeater.cc:644] Failed to heartbeat to 127.15.75.190:44007 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 02:18:24.283438 3282 heartbeater.cc:464] Registering TS with master...
I20260809 02:18:24.283595 3282 heartbeater.cc:511] Master 127.15.75.190:44007 requested a full tablet report, sending...
I20260809 02:18:24.284387 3282 heartbeater.cc:464] Registering TS with master...
I20260809 02:18:24.284535 3282 heartbeater.cc:511] Master 127.15.75.190:44007 requested a full tablet report, sending...
W20260809 02:18:24.285310 3282 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 02:18:24.425549 3287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:18:24.425957 3287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:18:24.426033 3287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:18:24.432142 3287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:18:24.432231 3287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:18:24.432327 3287 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.131
I20260809 02:18:24.440379 3287 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:36263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.15.75.148:44415/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab
--principal=kudu/127.15.75.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.131:44853
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.75.131
--webserver_port=38621
--tserver_master_addrs=127.15.75.190:44007
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3287
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.131
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:18:24.441843 3287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:18:24.443316 3287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 02:18:24.456846 3287 server_base.cc:1034] running on GCE node
W20260809 02:18:24.457036 3295 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:24.457357 3298 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:24.457597 3296 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:24.458101 3287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:18:24.458948 3287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:18:24.460135 3287 hybrid_clock.cc:648] HybridClock initialized: now 1786241904460106 us; error 43 us; skew 500 ppm
Aug 09 02:18:24 dist-test-slave-s11t krb5kdc[1124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241904, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:18:24.463965 3287 init.cc:377] Logged in from keytab as kudu/127.15.75.131@KRBTEST.COM (short username kudu)
I20260809 02:18:24.465162 3287 webserver.cc:460] Webserver started at http://127.15.75.131:38621/ using document root <none> and password file <none>
I20260809 02:18:24.465914 3287 fs_manager.cc:359] Metadata directory not provided
I20260809 02:18:24.466030 3287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 02:18:24 dist-test-slave-s11t krb5kdc[1124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241904, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.131@KRBTEST.COM for HTTP/127.15.75.148@KRBTEST.COM
I20260809 02:18:24.486419 3287 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 02:18:24.489696 3314 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:24.491272 3287 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260809 02:18:24.491444 3287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-2/wal
uuid: "b68bbc0c4f0c47479e75790e960df666"
format_stamp: "Formatted at 2026-08-09 02:18:23 on dist-test-slave-s11t"
server_key: "af445fbb9587283a73b080d137b37732"
server_key_iv: "1ee9bd1d6ba13a7fb90c1af6288a4451"
server_key_version: "kuduclusterkey@0"
I20260809 02:18:24.492170 3287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:18:24.518121 3287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:18:24.521826 3287 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:18:24.522047 3287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:18:24.522753 3287 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:18:24.523793 3287 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:18:24.523862 3287 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:24.523945 3287 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:18:24.523983 3287 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:24.535876 3287 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.131:44853
I20260809 02:18:24.535989 3490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.131:44853 every 8 connection(s)
I20260809 02:18:24.537212 3287 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
I20260809 02:18:24.544394 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 3287
Aug 09 02:18:24 dist-test-slave-s11t krb5kdc[1124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241904, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.131@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:18:24.555083 3057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:24.540927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.131:55609 (local address 127.15.75.190:44007)
0809 02:18:24.541072 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:24.541079 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:24.541491 (+ 412us) server_negotiation.cc:407] Connection header received
0809 02:18:24.542492 (+ 1001us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:24.542496 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:24.542557 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:24.542635 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:24.543990 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:24.544816 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:24.546540 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:24.546786 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:24.546896 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:24.549599 (+ 2703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:24.549603 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:24.549606 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:18:24.549633 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:24.552499 (+ 2866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:24.553083 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:24.553089 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:24.553093 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:24.553191 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:24.553656 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:24.553660 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:24.553661 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:24.553890 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:24.553984 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:24.554727 (+ 743us) server_negotiation.cc:299] Negotiation successful
0809 02:18:24.554873 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 02:18:24.556668 3493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:24.540793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:44007 (local address 127.15.75.131:55609)
0809 02:18:24.541326 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:24.541390 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:24.542274 (+ 884us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:24.542980 (+ 706us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:24.542990 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:24.543302 (+ 312us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:18:24.543769 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:24.543810 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:24.545018 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:24.545024 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:24.546420 (+ 1396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:24.546429 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:24.546957 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:24.546961 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:24.547137 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:24.547768 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:18:24.547785 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:18:24.549399 (+ 1614us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:18:24.552613 (+ 3214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:18:24.552623 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:18:24.552640 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:18:24.552979 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:18:24.553321 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:18:24.553325 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:18:24.553327 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:18:24.553532 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:18:24.554060 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:18:24.554066 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:18:24.554380 (+ 314us) client_negotiation.cc:769] Sending connection context
0809 02:18:24.554618 (+ 238us) client_negotiation.cc:240] Negotiation successful
0809 02:18:24.554955 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":156,"threads_started":1}
I20260809 02:18:24.558117 3491 heartbeater.cc:347] Connected to a master server at 127.15.75.190:44007
I20260809 02:18:24.558374 3491 heartbeater.cc:464] Registering TS with master...
I20260809 02:18:24.558950 3491 heartbeater.cc:511] Master 127.15.75.190:44007 requested a full tablet report, sending...
W20260809 02:18:24.560761 3491 heartbeater.cc:644] Failed to heartbeat to 127.15.75.190:44007 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 02:18:24.560890 3491 heartbeater.cc:464] Registering TS with master...
I20260809 02:18:24.561061 3491 heartbeater.cc:511] Master 127.15.75.190:44007 requested a full tablet report, sending...
I20260809 02:18:24.561883 3491 heartbeater.cc:464] Registering TS with master...
I20260809 02:18:24.562098 3491 heartbeater.cc:511] Master 127.15.75.190:44007 requested a full tablet report, sending...
I20260809 02:18:24.562484 3500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:24.545319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44038 (local address 127.15.75.190:44007)
0809 02:18:24.545644 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:24.545656 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:24.548775 (+ 3119us) server_negotiation.cc:407] Connection header received
0809 02:18:24.548824 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:24.548827 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:24.548888 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:24.548988 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:24.550183 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:24.551001 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:24.552265 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:24.552559 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:24.552663 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:24.554382 (+ 1719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:24.554386 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:24.554388 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:18:24.554413 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:24.556017 (+ 1604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:24.556451 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:24.556456 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:24.556458 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:24.556566 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:24.561050 (+ 4484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:24.561054 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:24.561056 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:24.561274 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:24.561468 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:24.562160 (+ 692us) server_negotiation.cc:299] Negotiation successful
0809 02:18:24.562315 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":87,"threads_started":1}
W20260809 02:18:24.563237 3491 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 09 02:18:24 dist-test-slave-s11t krb5kdc[1124](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786241904, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:18:24.972023 3053 heartbeater.cc:464] Registering TS with master...
I20260809 02:18:24.972333 3053 heartbeater.cc:511] Master 127.15.75.190:44007 requested a full tablet report, sending...
I20260809 02:18:24.973548 2722 ts_manager.cc:194] Registered new tserver with Master: f4a558685b5245aea197dacba4d07b07 (127.15.75.129:38959)
I20260809 02:18:24.975014 2722 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.75.129@KRBTEST.COM'} at 127.15.75.129:36949
I20260809 02:18:25.285547 3282 heartbeater.cc:464] Registering TS with master...
I20260809 02:18:25.285861 3282 heartbeater.cc:511] Master 127.15.75.190:44007 requested a full tablet report, sending...
I20260809 02:18:25.286670 2722 ts_manager.cc:194] Registered new tserver with Master: 37e0dba10c8944de86fbe4f8c414c43c (127.15.75.130:38111)
I20260809 02:18:25.287499 2722 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.75.130@KRBTEST.COM'} at 127.15.75.130:40771
Aug 09 02:18:25 dist-test-slave-s11t krb5kdc[1124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241904, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.190@KRBTEST.COM for HTTP/127.15.75.148@KRBTEST.COM
I20260809 02:18:25.563470 3491 heartbeater.cc:464] Registering TS with master...
I20260809 02:18:25.563766 3491 heartbeater.cc:511] Master 127.15.75.190:44007 requested a full tablet report, sending...
I20260809 02:18:25.564584 2722 ts_manager.cc:194] Registered new tserver with Master: b68bbc0c4f0c47479e75790e960df666 (127.15.75.131:44853)
I20260809 02:18:25.565385 2722 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.75.131@KRBTEST.COM'} at 127.15.75.131:55609
I20260809 02:18:25.567894 15662 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 02:18:25.581444 3523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:25.569907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44054 (local address 127.15.75.190:44007)
0809 02:18:25.570367 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:25.570371 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:25.570387 (+ 16us) server_negotiation.cc:407] Connection header received
0809 02:18:25.570426 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:25.570429 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:25.570473 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:25.570568 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:25.571448 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:25.572218 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:25.573447 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:25.573823 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:25.573931 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:25.576385 (+ 2454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:25.576390 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:25.576392 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:18:25.576419 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:25.578386 (+ 1967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:25.578913 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:25.578917 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:25.578919 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:25.579016 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:25.579344 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:25.579348 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:25.579350 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:25.579569 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:25.579768 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:25.581124 (+ 1356us) server_negotiation.cc:299] Negotiation successful
0809 02:18:25.581254 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":86,"threads_started":1}
I20260809 02:18:25.585846 2722 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44054:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:18:25.589564 2722 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:18:25.609792 3543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:25.601196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.131:44853 (local address 127.0.0.1:55166)
0809 02:18:25.604235 (+ 3039us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:25.604248 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:25.604337 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:25.604996 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:25.605003 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:25.605026 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:18:25.605256 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:25.605266 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:25.606682 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:25.606685 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:25.608013 (+ 1328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:25.608025 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:25.609288 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:25.609292 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:25.609397 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:25.609434 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 02:18:25.609540 (+ 106us) client_negotiation.cc:240] Negotiation successful
0809 02:18:25.609648 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2977,"thread_start_us":79,"threads_started":1}
I20260809 02:18:25.610363 3541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:25.601413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55166 (local address 127.15.75.131:44853)
0809 02:18:25.603357 (+ 1944us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:25.603365 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:25.604443 (+ 1078us) server_negotiation.cc:407] Connection header received
0809 02:18:25.604567 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:25.604573 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:25.604826 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:25.604986 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:18:25.605400 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:25.606545 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:25.608162 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:25.608969 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:25.609077 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:25.609222 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:25.610128 (+ 906us) server_negotiation.cc:299] Negotiation successful
0809 02:18:25.610230 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1833,"thread_start_us":122,"threads_started":1}
I20260809 02:18:25.613837 3398 tablet_service.cc:1467] Processing CreateTablet for tablet d1e0c1326236458a8d5f539299e9999e (DEFAULT_TABLE table=test-table [id=3e8127208de646cca7da01c29c4dc812]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:18:25.614951 3398 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1e0c1326236458a8d5f539299e9999e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:18:25.617933 3538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:25.599927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.129:38959 (local address 127.0.0.1:45284)
0809 02:18:25.603817 (+ 3890us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:25.603867 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:25.604000 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:25.606398 (+ 2398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:25.606404 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:25.606427 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:18:25.606527 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:25.606533 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:25.611833 (+ 5300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:25.611836 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:25.612973 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:25.612985 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:25.617496 (+ 4511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:25.617500 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:25.617601 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:25.617626 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 02:18:25.617702 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 02:18:25.617773 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3682,"thread_start_us":87,"threads_started":1}
I20260809 02:18:25.618500 3536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:25.601512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.130:38111 (local address 127.0.0.1:35374)
0809 02:18:25.608161 (+ 6649us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:25.608175 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:25.608268 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:25.608970 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:25.608973 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:25.608992 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:18:25.609071 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:25.609077 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:25.613114 (+ 4037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:25.613117 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:25.614235 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:25.614246 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:25.618087 (+ 3841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:25.618091 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:25.618190 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:25.618213 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 02:18:25.618301 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 02:18:25.618370 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6548,"thread_start_us":92,"threads_started":1}
I20260809 02:18:25.618901 3544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:25.601771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35374 (local address 127.15.75.130:38111)
0809 02:18:25.603536 (+ 1765us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:25.603542 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:25.608380 (+ 4838us) server_negotiation.cc:407] Connection header received
0809 02:18:25.608463 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:25.608468 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:25.608645 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:25.608820 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:18:25.609241 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:25.610300 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:25.614410 (+ 4110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:25.615207 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:25.615331 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:25.615475 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:25.618654 (+ 3179us) server_negotiation.cc:299] Negotiation successful
0809 02:18:25.618757 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1668,"thread_start_us":74,"threads_started":1}
I20260809 02:18:25.619649 3537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:25.600082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45284 (local address 127.15.75.129:38959)
0809 02:18:25.605786 (+ 5704us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:25.605793 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:25.605818 (+ 25us) server_negotiation.cc:407] Connection header received
0809 02:18:25.605890 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:25.605894 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:25.606078 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:25.606227 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:18:25.606674 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:25.610935 (+ 4261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:25.615543 (+ 4608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:25.617170 (+ 1627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:25.617293 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:25.617428 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:25.619411 (+ 1983us) server_negotiation.cc:299] Negotiation successful
0809 02:18:25.619512 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5585,"thread_start_us":122,"threads_started":1}
I20260809 02:18:25.622105 2900 tablet_service.cc:1467] Processing CreateTablet for tablet d1e0c1326236458a8d5f539299e9999e (DEFAULT_TABLE table=test-table [id=3e8127208de646cca7da01c29c4dc812]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:18:25.622105 3167 tablet_service.cc:1467] Processing CreateTablet for tablet d1e0c1326236458a8d5f539299e9999e (DEFAULT_TABLE table=test-table [id=3e8127208de646cca7da01c29c4dc812]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:18:25.623173 2900 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1e0c1326236458a8d5f539299e9999e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:18:25.623173 3167 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1e0c1326236458a8d5f539299e9999e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:18:25.629196 3546 tablet_bootstrap.cc:492] T d1e0c1326236458a8d5f539299e9999e P b68bbc0c4f0c47479e75790e960df666: Bootstrap starting.
I20260809 02:18:25.631567 3546 tablet_bootstrap.cc:654] T d1e0c1326236458a8d5f539299e9999e P b68bbc0c4f0c47479e75790e960df666: Neither blocks nor log segments found. Creating new log.
I20260809 02:18:25.632445 3546 log.cc:824] T d1e0c1326236458a8d5f539299e9999e P b68bbc0c4f0c47479e75790e960df666: Log is configured to *not* fsync() on all Append() calls
I20260809 02:18:25.635002 3549 tablet_bootstrap.cc:492] T d1e0c1326236458a8d5f539299e9999e P 37e0dba10c8944de86fbe4f8c414c43c: Bootstrap starting.
I20260809 02:18:25.637205 3549 tablet_bootstrap.cc:654] T d1e0c1326236458a8d5f539299e9999e P 37e0dba10c8944de86fbe4f8c414c43c: Neither blocks nor log segments found. Creating new log.
I20260809 02:18:25.637519 3547 tablet_bootstrap.cc:492] T d1e0c1326236458a8d5f539299e9999e P f4a558685b5245aea197dacba4d07b07: Bootstrap starting.
I20260809 02:18:25.638077 3549 log.cc:824] T d1e0c1326236458a8d5f539299e9999e P 37e0dba10c8944de86fbe4f8c414c43c: Log is configured to *not* fsync() on all Append() calls
I20260809 02:18:25.639643 3547 tablet_bootstrap.cc:654] T d1e0c1326236458a8d5f539299e9999e P f4a558685b5245aea197dacba4d07b07: Neither blocks nor log segments found. Creating new log.
I20260809 02:18:25.640455 3547 log.cc:824] T d1e0c1326236458a8d5f539299e9999e P f4a558685b5245aea197dacba4d07b07: Log is configured to *not* fsync() on all Append() calls
I20260809 02:18:25.640628 3549 tablet_bootstrap.cc:492] T d1e0c1326236458a8d5f539299e9999e P 37e0dba10c8944de86fbe4f8c414c43c: No bootstrap required, opened a new log
I20260809 02:18:25.640844 3549 ts_tablet_manager.cc:1397] T d1e0c1326236458a8d5f539299e9999e P 37e0dba10c8944de86fbe4f8c414c43c: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260809 02:18:25.642935 3547 tablet_bootstrap.cc:492] T d1e0c1326236458a8d5f539299e9999e P f4a558685b5245aea197dacba4d07b07: No bootstrap required, opened a new log
I20260809 02:18:25.643114 3547 ts_tablet_manager.cc:1397] T d1e0c1326236458a8d5f539299e9999e P f4a558685b5245aea197dacba4d07b07: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 02:18:25.644961 3549 raft_consensus.cc:348] T d1e0c1326236458a8d5f539299e9999e P 37e0dba10c8944de86fbe4f8c414c43c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4a558685b5245aea197dacba4d07b07" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 38959 } } peers { permanent_uuid: "37e0dba10c8944de86fbe4f8c414c43c" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 38111 } } peers { permanent_uuid: "b68bbc0c4f0c47479e75790e960df666" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 44853 } }
I20260809 02:18:25.645224 3549 raft_consensus.cc:374] T d1e0c1326236458a8d5f539299e9999e P 37e0dba10c8944de86fbe4f8c414c43c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:18:25.645298 3549 raft_consensus.cc:729] T d1e0c1326236458a8d5f539299e9999e P 37e0dba10c8944de86fbe4f8c414c43c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37e0dba10c8944de86fbe4f8c414c43c, State: Initialized, Role: FOLLOWER
I20260809 02:18:25.646703 3549 consensus_queue.cc:260] T d1e0c1326236458a8d5f539299e9999e P 37e0dba10c8944de86fbe4f8c414c43c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4a558685b5245aea197dacba4d07b07" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 38959 } } peers { permanent_uuid: "37e0dba10c8944de86fbe4f8c414c43c" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 38111 } } peers { permanent_uuid: "b68bbc0c4f0c47479e75790e960df666" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 44853 } }
I20260809 02:18:25.646929 3547 raft_consensus.cc:348] T d1e0c1326236458a8d5f539299e9999e P f4a558685b5245aea197dacba4d07b07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4a558685b5245aea197dacba4d07b07" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 38959 } } peers { permanent_uuid: "37e0dba10c8944de86fbe4f8c414c43c" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 38111 } } peers { permanent_uuid: "b68bbc0c4f0c47479e75790e960df666" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 44853 } }
I20260809 02:18:25.647161 3547 raft_consensus.cc:374] T d1e0c1326236458a8d5f539299e9999e P f4a558685b5245aea197dacba4d07b07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:18:25.647207 3547 raft_consensus.cc:729] T d1e0c1326236458a8d5f539299e9999e P f4a558685b5245aea197dacba4d07b07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4a558685b5245aea197dacba4d07b07, State: Initialized, Role: FOLLOWER
I20260809 02:18:25.647643 3547 consensus_queue.cc:260] T d1e0c1326236458a8d5f539299e9999e P f4a558685b5245aea197dacba4d07b07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4a558685b5245aea197dacba4d07b07" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 38959 } } peers { permanent_uuid: "37e0dba10c8944de86fbe4f8c414c43c" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 38111 } } peers { permanent_uuid: "b68bbc0c4f0c47479e75790e960df666" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 44853 } }
I20260809 02:18:25.647746 3549 ts_tablet_manager.cc:1428] T d1e0c1326236458a8d5f539299e9999e P 37e0dba10c8944de86fbe4f8c414c43c: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260809 02:18:25.648092 3282 heartbeater.cc:503] Master 127.15.75.190:44007 was elected leader, sending a full tablet report...
I20260809 02:18:25.648561 3547 ts_tablet_manager.cc:1428] T d1e0c1326236458a8d5f539299e9999e P f4a558685b5245aea197dacba4d07b07: Time spent starting tablet: real 0.005s user 0.007s sys 0.000s
I20260809 02:18:25.648900 3053 heartbeater.cc:503] Master 127.15.75.190:44007 was elected leader, sending a full tablet report...
I20260809 02:18:25.656726 3546 tablet_bootstrap.cc:492] T d1e0c1326236458a8d5f539299e9999e P b68bbc0c4f0c47479e75790e960df666: No bootstrap required, opened a new log
I20260809 02:18:25.656967 3546 ts_tablet_manager.cc:1397] T d1e0c1326236458a8d5f539299e9999e P b68bbc0c4f0c47479e75790e960df666: Time spent bootstrapping tablet: real 0.028s user 0.006s sys 0.000s
I20260809 02:18:25.660981 3546 raft_consensus.cc:348] T d1e0c1326236458a8d5f539299e9999e P b68bbc0c4f0c47479e75790e960df666 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4a558685b5245aea197dacba4d07b07" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 38959 } } peers { permanent_uuid: "37e0dba10c8944de86fbe4f8c414c43c" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 38111 } } peers { permanent_uuid: "b68bbc0c4f0c47479e75790e960df666" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 44853 } }
I20260809 02:18:25.661248 3546 raft_consensus.cc:374] T d1e0c1326236458a8d5f539299e9999e P b68bbc0c4f0c47479e75790e960df666 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:18:25.661303 3546 raft_consensus.cc:729] T d1e0c1326236458a8d5f539299e9999e P b68bbc0c4f0c47479e75790e960df666 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b68bbc0c4f0c47479e75790e960df666, State: Initialized, Role: FOLLOWER
Aug 09 02:18:25 dist-test-slave-s11t krb5kdc[1124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241904, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.190@KRBTEST.COM for HTTP/127.15.75.148@KRBTEST.COM
I20260809 02:18:25.661725 3546 consensus_queue.cc:260] T d1e0c1326236458a8d5f539299e9999e P b68bbc0c4f0c47479e75790e960df666 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4a558685b5245aea197dacba4d07b07" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 38959 } } peers { permanent_uuid: "37e0dba10c8944de86fbe4f8c414c43c" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 38111 } } peers { permanent_uuid: "b68bbc0c4f0c47479e75790e960df666" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 44853 } }
I20260809 02:18:25.662726 3546 ts_tablet_manager.cc:1428] T d1e0c1326236458a8d5f539299e9999e P b68bbc0c4f0c47479e75790e960df666: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260809 02:18:25.663074 3491 heartbeater.cc:503] Master 127.15.75.190:44007 was elected leader, sending a full tablet report...
W20260809 02:18:25.686784 3054 tablet.cc:2369] T d1e0c1326236458a8d5f539299e9999e P f4a558685b5245aea197dacba4d07b07: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 02:18:25.750452 3283 tablet.cc:2369] T d1e0c1326236458a8d5f539299e9999e P 37e0dba10c8944de86fbe4f8c414c43c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 02:18:25.791164 3492 tablet.cc:2369] T d1e0c1326236458a8d5f539299e9999e P b68bbc0c4f0c47479e75790e960df666: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:18:25.937780 3559 raft_consensus.cc:482] T d1e0c1326236458a8d5f539299e9999e P f4a558685b5245aea197dacba4d07b07 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:18:25.938020 3559 raft_consensus.cc:504] T d1e0c1326236458a8d5f539299e9999e P f4a558685b5245aea197dacba4d07b07 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4a558685b5245aea197dacba4d07b07" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 38959 } } peers { permanent_uuid: "37e0dba10c8944de86fbe4f8c414c43c" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 38111 } } peers { permanent_uuid: "b68bbc0c4f0c47479e75790e960df666" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 44853 } }
I20260809 02:18:25.939440 3559 leader_election.cc:288] T d1e0c1326236458a8d5f539299e9999e P f4a558685b5245aea197dacba4d07b07 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 37e0dba10c8944de86fbe4f8c414c43c (127.15.75.130:38111), b68bbc0c4f0c47479e75790e960df666 (127.15.75.131:44853)
I20260809 02:18:25.944207 3575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:25.939680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.131:44853 (local address 127.15.75.129:60657)
0809 02:18:25.940273 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:25.940286 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:25.940419 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:25.940764 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:25.940767 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:25.940787 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:18:25.940896 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:25.940902 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:25.941922 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:25.941924 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:25.942949 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:25.942959 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:25.943772 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:25.943776 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:25.943867 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:25.943895 (+ 28us) client_negotiation.cc:769] Sending connection context
0809 02:18:25.943974 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 02:18:25.944041 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"mutex_wait_us":278,"thread_start_us":96,"threads_started":1}
I20260809 02:18:25.944217 3541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:25.939881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.129:60657 (local address 127.15.75.131:44853)
0809 02:18:25.940055 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:25.940061 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:25.940355 (+ 294us) server_negotiation.cc:407] Connection header received
0809 02:18:25.940531 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:25.940535 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:25.940587 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:25.940656 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:18:25.941034 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:25.941797 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:25.943073 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:25.943640 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:25.943763 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:25.943816 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:25.944008 (+ 192us) server_negotiation.cc:299] Negotiation successful
0809 02:18:25.944087 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 02:18:25.944609 3544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:25.939896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.129:51093 (local address 127.15.75.130:38111)
0809 02:18:25.940151 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:25.940156 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:25.940229 (+ 73us) server_negotiation.cc:407] Connection header received
0809 02:18:25.940403 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:25.940407 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:25.940461 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:25.940556 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:18:25.940872 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:25.941729 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:25.943355 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:25.943928 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:25.944059 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:25.944101 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:25.944409 (+ 308us) server_negotiation.cc:299] Negotiation successful
0809 02:18:25.944493 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184}
I20260809 02:18:25.945055 3574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:25.939680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.130:38111 (local address 127.15.75.129:51093)
0809 02:18:25.940118 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:25.940136 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:25.940260 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:25.940560 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:25.940562 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:25.940577 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:18:25.940726 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:25.940734 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:25.941854 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:25.941858 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:25.943199 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:25.943211 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:25.944162 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:25.944166 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:25.944255 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:25.944281 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 02:18:25.944835 (+ 554us) client_negotiation.cc:240] Negotiation successful
0809 02:18:25.944919 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":367,"mutex_wait_us":65,"thread_start_us":144,"threads_started":1}
I20260809 02:18:25.945199 3426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1e0c1326236458a8d5f539299e9999e" candidate_uuid: "f4a558685b5245aea197dacba4d07b07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b68bbc0c4f0c47479e75790e960df666" is_pre_election: true
I20260809 02:18:25.945502 3426 raft_consensus.cc:2454] T d1e0c1326236458a8d5f539299e9999e P b68bbc0c4f0c47479e75790e960df666 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4a558685b5245aea197dacba4d07b07 in term 0.
I20260809 02:18:25.945710 3205 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1e0c1326236458a8d5f539299e9999e" candidate_uuid: "f4a558685b5245aea197dacba4d07b07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37e0dba10c8944de86fbe4f8c414c43c" is_pre_election: true
I20260809 02:18:25.946017 3205 raft_consensus.cc:2454] T d1e0c1326236458a8d5f539299e9999e P 37e0dba10c8944de86fbe4f8c414c43c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4a558685b5245aea197dacba4d07b07 in term 0.
I20260809 02:18:25.946000 2863 leader_election.cc:302] T d1e0c1326236458a8d5f539299e9999e P f4a558685b5245aea197dacba4d07b07 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b68bbc0c4f0c47479e75790e960df666, f4a558685b5245aea197dacba4d07b07; no voters:
I20260809 02:18:25.946274 3559 raft_consensus.cc:2781] T d1e0c1326236458a8d5f539299e9999e P f4a558685b5245aea197dacba4d07b07 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:18:25.946380 3559 raft_consensus.cc:482] T d1e0c1326236458a8d5f539299e9999e P f4a558685b5245aea197dacba4d07b07 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:18:25.946427 3559 raft_consensus.cc:3037] T d1e0c1326236458a8d5f539299e9999e P f4a558685b5245aea197dacba4d07b07 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:18:25.947546 3559 raft_consensus.cc:504] T d1e0c1326236458a8d5f539299e9999e P f4a558685b5245aea197dacba4d07b07 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4a558685b5245aea197dacba4d07b07" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 38959 } } peers { permanent_uuid: "37e0dba10c8944de86fbe4f8c414c43c" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 38111 } } peers { permanent_uuid: "b68bbc0c4f0c47479e75790e960df666" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 44853 } }
I20260809 02:18:25.947857 3559 leader_election.cc:288] T d1e0c1326236458a8d5f539299e9999e P f4a558685b5245aea197dacba4d07b07 [CANDIDATE]: Term 1 election: Requested vote from peers 37e0dba10c8944de86fbe4f8c414c43c (127.15.75.130:38111), b68bbc0c4f0c47479e75790e960df666 (127.15.75.131:44853)
I20260809 02:18:25.948201 3205 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1e0c1326236458a8d5f539299e9999e" candidate_uuid: "f4a558685b5245aea197dacba4d07b07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37e0dba10c8944de86fbe4f8c414c43c"
I20260809 02:18:25.948339 3205 raft_consensus.cc:3037] T d1e0c1326236458a8d5f539299e9999e P 37e0dba10c8944de86fbe4f8c414c43c [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:18:25.948387 3426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1e0c1326236458a8d5f539299e9999e" candidate_uuid: "f4a558685b5245aea197dacba4d07b07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b68bbc0c4f0c47479e75790e960df666"
I20260809 02:18:25.948509 3426 raft_consensus.cc:3037] T d1e0c1326236458a8d5f539299e9999e P b68bbc0c4f0c47479e75790e960df666 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:18:25.949656 3205 raft_consensus.cc:2454] T d1e0c1326236458a8d5f539299e9999e P 37e0dba10c8944de86fbe4f8c414c43c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4a558685b5245aea197dacba4d07b07 in term 1.
I20260809 02:18:25.949806 3426 raft_consensus.cc:2454] T d1e0c1326236458a8d5f539299e9999e P b68bbc0c4f0c47479e75790e960df666 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4a558685b5245aea197dacba4d07b07 in term 1.
I20260809 02:18:25.950030 2865 leader_election.cc:302] T d1e0c1326236458a8d5f539299e9999e P f4a558685b5245aea197dacba4d07b07 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37e0dba10c8944de86fbe4f8c414c43c, f4a558685b5245aea197dacba4d07b07; no voters:
I20260809 02:18:25.950233 3559 raft_consensus.cc:2781] T d1e0c1326236458a8d5f539299e9999e P f4a558685b5245aea197dacba4d07b07 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:18:25.950498 3559 raft_consensus.cc:686] T d1e0c1326236458a8d5f539299e9999e P f4a558685b5245aea197dacba4d07b07 [term 1 LEADER]: Becoming Leader. State: Replica: f4a558685b5245aea197dacba4d07b07, State: Running, Role: LEADER
I20260809 02:18:25.950901 3559 consensus_queue.cc:237] T d1e0c1326236458a8d5f539299e9999e P f4a558685b5245aea197dacba4d07b07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4a558685b5245aea197dacba4d07b07" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 38959 } } peers { permanent_uuid: "37e0dba10c8944de86fbe4f8c414c43c" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 38111 } } peers { permanent_uuid: "b68bbc0c4f0c47479e75790e960df666" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 44853 } }
I20260809 02:18:25.954257 2721 catalog_manager.cc:5495] T d1e0c1326236458a8d5f539299e9999e P f4a558685b5245aea197dacba4d07b07 reported cstate change: term changed from 0 to 1, leader changed from <none> to f4a558685b5245aea197dacba4d07b07 (127.15.75.129). New cstate: current_term: 1 leader_uuid: "f4a558685b5245aea197dacba4d07b07" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4a558685b5245aea197dacba4d07b07" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 38959 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "37e0dba10c8944de86fbe4f8c414c43c" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 38111 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b68bbc0c4f0c47479e75790e960df666" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 44853 } health_report { overall_health: UNKNOWN } } }
I20260809 02:18:25.976590 3537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:25.972804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45298 (local address 127.15.75.129:38959)
0809 02:18:25.972934 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:25.972938 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:25.973001 (+ 63us) server_negotiation.cc:407] Connection header received
0809 02:18:25.973201 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:25.973204 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:25.973271 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:25.973385 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:18:25.973638 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:25.974152 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:25.975052 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:25.975305 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:25.975414 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:25.975775 (+ 361us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:18:25.976199 (+ 424us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:18:25.976330 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:25.976393 (+ 63us) server_negotiation.cc:299] Negotiation successful
0809 02:18:25.976447 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 02:18:25.982527 3205 raft_consensus.cc:1261] T d1e0c1326236458a8d5f539299e9999e P 37e0dba10c8944de86fbe4f8c414c43c [term 1 FOLLOWER]: Refusing update from remote peer f4a558685b5245aea197dacba4d07b07: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:18:25.982745 3426 raft_consensus.cc:1261] T d1e0c1326236458a8d5f539299e9999e P b68bbc0c4f0c47479e75790e960df666 [term 1 FOLLOWER]: Refusing update from remote peer f4a558685b5245aea197dacba4d07b07: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:18:25.983311 3576 consensus_queue.cc:1035] T d1e0c1326236458a8d5f539299e9999e P f4a558685b5245aea197dacba4d07b07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "37e0dba10c8944de86fbe4f8c414c43c" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 38111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:18:25.983563 3559 consensus_queue.cc:1035] T d1e0c1326236458a8d5f539299e9999e P f4a558685b5245aea197dacba4d07b07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b68bbc0c4f0c47479e75790e960df666" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 44853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:18:25.992799 3585 mvcc.cc:205] Tried to move back new op lower bound from 7316446846896529408 to 7316446846777765888. Current Snapshot: MvccSnapshot[applied={T|T < 7316446846896529408}]
I20260809 02:18:25.993235 3581 mvcc.cc:205] Tried to move back new op lower bound from 7316446846896529408 to 7316446846777765888. Current Snapshot: MvccSnapshot[applied={T|T < 7316446846896529408}]
I20260809 02:18:26.003942 2722 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44054:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:18:26.004141 2722 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44054:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:18:26.006687 2722 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 61b8e6f14b0a46f99eb43653e7b9b450: Sending DeleteTablet for 3 replicas of tablet d1e0c1326236458a8d5f539299e9999e
I20260809 02:18:26.007694 3167 tablet_service.cc:1514] Processing DeleteTablet for tablet d1e0c1326236458a8d5f539299e9999e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:18:25 UTC) from {username='kudu', principal='kudu/127.15.75.190@KRBTEST.COM'} at 127.0.0.1:35374
I20260809 02:18:26.007690 2900 tablet_service.cc:1514] Processing DeleteTablet for tablet d1e0c1326236458a8d5f539299e9999e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:18:25 UTC) from {username='kudu', principal='kudu/127.15.75.190@KRBTEST.COM'} at 127.0.0.1:45284
I20260809 02:18:26.008193 3398 tablet_service.cc:1514] Processing DeleteTablet for tablet d1e0c1326236458a8d5f539299e9999e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:18:25 UTC) from {username='kudu', principal='kudu/127.15.75.190@KRBTEST.COM'} at 127.0.0.1:55166
I20260809 02:18:26.008891 3597 tablet_replica.cc:330] T d1e0c1326236458a8d5f539299e9999e P b68bbc0c4f0c47479e75790e960df666: stopping tablet replica
I20260809 02:18:26.009629 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 2785
I20260809 02:18:26.021201 3597 raft_consensus.cc:2229] T d1e0c1326236458a8d5f539299e9999e P b68bbc0c4f0c47479e75790e960df666 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:18:26.021634 3597 raft_consensus.cc:2258] T d1e0c1326236458a8d5f539299e9999e P b68bbc0c4f0c47479e75790e960df666 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:18:26.022078 3596 tablet_replica.cc:330] T d1e0c1326236458a8d5f539299e9999e P 37e0dba10c8944de86fbe4f8c414c43c: stopping tablet replica
I20260809 02:18:26.022415 3596 raft_consensus.cc:2229] T d1e0c1326236458a8d5f539299e9999e P 37e0dba10c8944de86fbe4f8c414c43c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:18:26.022738 3596 raft_consensus.cc:2258] T d1e0c1326236458a8d5f539299e9999e P 37e0dba10c8944de86fbe4f8c414c43c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:18:26.023547 3597 ts_tablet_manager.cc:1905] T d1e0c1326236458a8d5f539299e9999e P b68bbc0c4f0c47479e75790e960df666: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:18:26.024303 3596 ts_tablet_manager.cc:1905] T d1e0c1326236458a8d5f539299e9999e P 37e0dba10c8944de86fbe4f8c414c43c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:18:26.026564 3597 ts_tablet_manager.cc:1918] T d1e0c1326236458a8d5f539299e9999e P b68bbc0c4f0c47479e75790e960df666: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:18:26.026675 3597 log.cc:1196] T d1e0c1326236458a8d5f539299e9999e P b68bbc0c4f0c47479e75790e960df666: Deleting WAL directory at /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-2/wal/wals/d1e0c1326236458a8d5f539299e9999e
I20260809 02:18:26.027052 3597 ts_tablet_manager.cc:1939] T d1e0c1326236458a8d5f539299e9999e P b68bbc0c4f0c47479e75790e960df666: Deleting consensus metadata
I20260809 02:18:26.027355 3596 ts_tablet_manager.cc:1918] T d1e0c1326236458a8d5f539299e9999e P 37e0dba10c8944de86fbe4f8c414c43c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:18:26.027510 3596 log.cc:1196] T d1e0c1326236458a8d5f539299e9999e P 37e0dba10c8944de86fbe4f8c414c43c: Deleting WAL directory at /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241741788589-15662-0/minicluster-data/ts-1/wal/wals/d1e0c1326236458a8d5f539299e9999e
I20260809 02:18:26.027900 3596 ts_tablet_manager.cc:1939] T d1e0c1326236458a8d5f539299e9999e P 37e0dba10c8944de86fbe4f8c414c43c: Deleting consensus metadata
I20260809 02:18:26.028615 2700 catalog_manager.cc:4841] TS b68bbc0c4f0c47479e75790e960df666 (127.15.75.131:44853): tablet d1e0c1326236458a8d5f539299e9999e (table test-table [id=3e8127208de646cca7da01c29c4dc812]) successfully deleted
I20260809 02:18:26.029577 2703 catalog_manager.cc:4841] TS 37e0dba10c8944de86fbe4f8c414c43c (127.15.75.130:38111): tablet d1e0c1326236458a8d5f539299e9999e (table test-table [id=3e8127208de646cca7da01c29c4dc812]) successfully deleted
W20260809 02:18:26.030406 2703 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.75.129:38959 (error 108)
I20260809 02:18:26.031433 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 3055
I20260809 02:18:26.031558 3543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:26.031252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.129:38959 (local address 127.0.0.1:45314)
0809 02:18:26.031401 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:26.031473 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.129:38959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260809 02:18:26.031706 2703 catalog_manager.cc:4584] TS f4a558685b5245aea197dacba4d07b07 (127.15.75.129:38959): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d1e0c1326236458a8d5f539299e9999e: Network error: Client connection negotiation failed: client connection to 127.15.75.129:38959: connect: Connection refused (error 111)
I20260809 02:18:26.049727 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 3287
I20260809 02:18:26.062556 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 2675
I20260809 02:18:26.081311 15662 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 02:18:26.425480 15662 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 02:18:26.425593 15662 mini_ranger.cc:66] Stopping Ranger...
I20260809 02:18:26.779953 15662 mini_ranger.cc:68] Stopped Ranger
2026-08-09 02:18:26.780 UTC [1175] LOG: received smart shutdown request
2026-08-09 02:18:26.784 UTC [1175] LOG: background worker "logical replication launcher" (PID 1188) exited with exit code 1
2026-08-09 02:18:26.784 UTC [1183] LOG: shutting down
2026-08-09 02:18:26.929 UTC [1175] LOG: database system is shut down
2026-08-09T02:18:26Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (50133 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241741788589-15662-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:18:27 dist-test-slave-s11t krb5kdc[3611](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:18:27 dist-test-slave-s11t krb5kdc[3611](info): set up 2 sockets
Aug 09 02:18:27 dist-test-slave-s11t krb5kdc[3611](info): commencing operation
krb5kdc: starting...
W20260809 02:18:28.295639 15662 mini_kdc.cc:121] Time spent starting KDC: real 1.222s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Aug 09 02:18:28 dist-test-slave-s11t krb5kdc[3611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241908, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:18:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:18:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.75.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab.
Entry for principal kudu/127.15.75.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab.
I20260809 02:18:28.417649 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241741788589-15662-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241741788589-15662-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241741788589-15662-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:40747
--webserver_interface=127.15.75.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab
--principal=kudu/127.15.75.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:45851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.75.190:40747,127.15.75.189:33671,127.15.75.188:34753
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:18:28.526751 3637 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:18:28.526988 3637 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:18:28.527027 3637 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:18:28.530840 3637 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:18:28.530887 3637 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:18:28.530912 3637 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:18:28.530929 3637 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:18:28.530944 3637 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:18:28.536136 3637 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:45851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241741788589-15662-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241741788589-15662-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.75.190:40747,127.15.75.189:33671,127.15.75.188:34753
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab
--principal=kudu/127.15.75.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:40747
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.75.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3637
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241741788589-15662-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:18:28.537015 3637 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:18:28.537910 3637 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:18:28.546080 3646 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:28.546700 3637 server_base.cc:1034] running on GCE node
W20260809 02:18:28.546607 3649 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:28.546602 3645 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:28.547384 3637 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:18:28.548410 3637 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:18:28.549594 3637 hybrid_clock.cc:648] HybridClock initialized: now 1786241908549551 us; error 55 us; skew 500 ppm
Aug 09 02:18:28 dist-test-slave-s11t krb5kdc[3611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241908, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:18:28.552359 3637 init.cc:377] Logged in from keytab as kudu/127.15.75.190@KRBTEST.COM (short username kudu)
I20260809 02:18:28.553277 3637 webserver.cc:460] Webserver started at http://127.15.75.190:44119/ using document root <none> and password file <none>
I20260809 02:18:28.553812 3637 fs_manager.cc:359] Metadata directory not provided
I20260809 02:18:28.553903 3637 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:18:28.554126 3637 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:18:28.556062 3637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241741788589-15662-0/minicluster-data/master-0/data/instance:
uuid: "6e8cd179cc854127a506990625b56e38"
format_stamp: "Formatted at 2026-08-09 02:18:28 on dist-test-slave-s11t"
server_key: "f4689432b808313c22be982af0660abf"
server_key_iv: "e65571dc9aa5c54f84614119064592eb"
server_key_version: "encryptionkey@0"
I20260809 02:18:28.556622 3637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241741788589-15662-0/minicluster-data/master-0/wal/instance:
uuid: "6e8cd179cc854127a506990625b56e38"
format_stamp: "Formatted at 2026-08-09 02:18:28 on dist-test-slave-s11t"
server_key: "f4689432b808313c22be982af0660abf"
server_key_iv: "e65571dc9aa5c54f84614119064592eb"
server_key_version: "encryptionkey@0"
I20260809 02:18:28.560078 3637 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260809 02:18:28.562461 3660 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:28.563695 3637 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260809 02:18:28.563834 3637 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241741788589-15662-0/minicluster-data/master-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241741788589-15662-0/minicluster-data/master-0/wal
uuid: "6e8cd179cc854127a506990625b56e38"
format_stamp: "Formatted at 2026-08-09 02:18:28 on dist-test-slave-s11t"
server_key: "f4689432b808313c22be982af0660abf"
server_key_iv: "e65571dc9aa5c54f84614119064592eb"
server_key_version: "encryptionkey@0"
I20260809 02:18:28.563946 3637 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241741788589-15662-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241741788589-15662-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241741788589-15662-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:18:28.582732 3637 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:18:28.586019 3637 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:18:28.586253 3637 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:18:28.593745 3637 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.190:40747
I20260809 02:18:28.593777 3743 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.190:40747 every 8 connection(s)
I20260809 02:18:28.594666 3637 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
I20260809 02:18:28.597445 3744 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:18:28.601486 3744 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.75.190" port: 40747 } has no permanent_uuid. Determining permanent_uuid...
I20260809 02:18:28.603547 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 3637
I20260809 02:18:28.603682 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241741788589-15662-0/minicluster-data/master-0/wal/instance
I20260809 02:18:28.603971 15662 external_mini_cluster.cc:1428] Setting key de42be1892221b160894b200da4c2095
WARNING: no policy specified for kudu/127.15.75.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.189@KRBTEST.COM" created.
Aug 09 02:18:28 dist-test-slave-s11t krb5kdc[3611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241908, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.190@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:18:28.621627 3747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:28.603996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44382 (local address 127.15.75.190:40747)
0809 02:18:28.604503 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:28.604523 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:28.604834 (+ 311us) server_negotiation.cc:407] Connection header received
0809 02:18:28.606378 (+ 1544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:28.606387 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:28.606627 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:28.606837 (+ 210us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:28.608268 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:28.609614 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:28.611454 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:28.611966 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:28.612269 (+ 303us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:28.616393 (+ 4124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:28.616398 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:28.616407 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:18:28.616427 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:28.618533 (+ 2106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:28.619021 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:28.619026 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:28.619028 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:28.619144 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:28.619465 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:28.619471 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:28.619472 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:28.619839 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:28.619979 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:28.620365 (+ 386us) server_negotiation.cc:299] Negotiation successful
0809 02:18:28.620609 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":332,"spinlock_wait_cycles":1280,"thread_start_us":162,"threads_started":1}
I20260809 02:18:28.621627 3748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:28.603996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:40747 (local address 127.0.0.1:44382)
0809 02:18:28.604630 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:28.604679 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:28.605947 (+ 1268us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:28.606911 (+ 964us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:28.606922 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:28.607569 (+ 647us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:18:28.608023 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:28.608043 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:28.609817 (+ 1774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:28.609822 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:28.611226 (+ 1404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:28.611242 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:28.612196 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:28.612202 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:28.612323 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:28.613187 (+ 864us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:18:28.613226 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:18:28.616215 (+ 2989us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:18:28.618648 (+ 2433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:18:28.618659 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:18:28.618668 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:18:28.618904 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:18:28.619250 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:18:28.619253 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:18:28.619254 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:18:28.619371 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:18:28.619963 (+ 592us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:18:28.619969 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:18:28.620121 (+ 152us) client_negotiation.cc:769] Sending connection context
0809 02:18:28.620289 (+ 168us) client_negotiation.cc:240] Negotiation successful
0809 02:18:28.620696 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"mutex_wait_us":110,"spinlock_wait_cycles":17920,"thread_start_us":215,"threads_started":1}
I20260809 02:18:28.624294 3744 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.75.189" port: 33671 } has no permanent_uuid. Determining permanent_uuid...
I20260809 02:18:28.625212 3748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:28.624951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.189:33671 (local address 127.0.0.1:39104)
0809 02:18:28.625083 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:28.625152 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.189:33671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260809 02:18:28.625454 3666 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.75.189:33671: connect: Connection refused (error 111)
I20260809 02:18:28.626878 3748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:28.626520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.189:33671 (local address 127.0.0.1:39114)
0809 02:18:28.626733 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:28.626792 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.189:33671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260809 02:18:28.627960 3744 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.75.189:33671: Network error: Client connection negotiation failed: client connection to 127.15.75.189:33671: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.189.keytab.
Entry for principal kudu/127.15.75.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.189.keytab.
I20260809 02:18:28.631461 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241741788589-15662-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241741788589-15662-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241741788589-15662-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241741788589-15662-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.189:33671
--webserver_interface=127.15.75.189
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.189.keytab
--principal=kudu/127.15.75.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:45851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.75.190:40747,127.15.75.189:33671,127.15.75.188:34753
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 02:18:28.677261 3744 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.75.189" port: 33671 } attempt: 1
I20260809 02:18:28.678138 3748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:28.677831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.189:33671 (local address 127.0.0.1:39124)
0809 02:18:28.677986 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:28.678061 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.189:33671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260809 02:18:28.678959 3748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:28.678734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.189:33671 (local address 127.0.0.1:39132)
0809 02:18:28.678860 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:28.678906 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.189:33671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260809 02:18:28.679126 3744 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.75.189:33671: Network error: Client connection negotiation failed: client connection to 127.15.75.189:33671: connect: Connection refused (error 111)
W20260809 02:18:28.741864 3753 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:18:28.742095 3753 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:18:28.742133 3753 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:18:28.746043 3753 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:18:28.746093 3753 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:18:28.746111 3753 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:18:28.746129 3753 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:18:28.746143 3753 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:18:28.747314 3744 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.75.189" port: 33671 } attempt: 2
I20260809 02:18:28.748139 3748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:28.747853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.189:33671 (local address 127.0.0.1:39148)
0809 02:18:28.748003 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:28.748064 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.189:33671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260809 02:18:28.748966 3748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:28.748677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.189:33671 (local address 127.0.0.1:39160)
0809 02:18:28.748865 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:28.748912 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.189:33671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
W20260809 02:18:28.749176 3744 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.75.189:33671: Network error: Client connection negotiation failed: client connection to 127.15.75.189:33671: connect: Connection refused (error 111)
I20260809 02:18:28.751783 3753 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:45851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241741788589-15662-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241741788589-15662-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.15.75.190:40747,127.15.75.189:33671,127.15.75.188:34753
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.189.keytab
--principal=kudu/127.15.75.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.189:33671
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241741788589-15662-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.15.75.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3753
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241741788589-15662-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:18:28.752790 3753 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:18:28.753723 3753 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:18:28.761981 3761 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:28.762005 3764 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:28.761981 3760 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:28.762337 3753 server_base.cc:1034] running on GCE node
I20260809 02:18:28.762925 3753 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:18:28.763914 3753 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:18:28.765102 3753 hybrid_clock.cc:648] HybridClock initialized: now 1786241908765089 us; error 23 us; skew 500 ppm
Aug 09 02:18:28 dist-test-slave-s11t krb5kdc[3611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241908, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:18:28.767771 3753 init.cc:377] Logged in from keytab as kudu/127.15.75.189@KRBTEST.COM (short username kudu)
I20260809 02:18:28.768776 3753 webserver.cc:460] Webserver started at http://127.15.75.189:37759/ using document root <none> and password file <none>
I20260809 02:18:28.769304 3753 fs_manager.cc:359] Metadata directory not provided
I20260809 02:18:28.769416 3753 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:18:28.769635 3753 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:18:28.771595 3753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241741788589-15662-0/minicluster-data/master-1/data/instance:
uuid: "c70d1718ca4e42ec8b887716d3cccd56"
format_stamp: "Formatted at 2026-08-09 02:18:28 on dist-test-slave-s11t"
server_key: "aea199a60f19506572ced78a2e89a526"
server_key_iv: "dddd678a7df8cd093de46deb5ad006ee"
server_key_version: "encryptionkey@0"
I20260809 02:18:28.772219 3753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241741788589-15662-0/minicluster-data/master-1/wal/instance:
uuid: "c70d1718ca4e42ec8b887716d3cccd56"
format_stamp: "Formatted at 2026-08-09 02:18:28 on dist-test-slave-s11t"
server_key: "aea199a60f19506572ced78a2e89a526"
server_key_iv: "dddd678a7df8cd093de46deb5ad006ee"
server_key_version: "encryptionkey@0"
I20260809 02:18:28.775715 3753 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260809 02:18:28.778052 3778 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:28.779208 3753 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 02:18:28.779358 3753 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241741788589-15662-0/minicluster-data/master-1/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241741788589-15662-0/minicluster-data/master-1/wal
uuid: "c70d1718ca4e42ec8b887716d3cccd56"
format_stamp: "Formatted at 2026-08-09 02:18:28 on dist-test-slave-s11t"
server_key: "aea199a60f19506572ced78a2e89a526"
server_key_iv: "dddd678a7df8cd093de46deb5ad006ee"
server_key_version: "encryptionkey@0"
I20260809 02:18:28.779479 3753 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241741788589-15662-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241741788589-15662-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241741788589-15662-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:18:28.792392 3753 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:18:28.794965 3753 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:18:28.795161 3753 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:18:28.802256 3753 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.189:33671
I20260809 02:18:28.802270 3855 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.189:33671 every 8 connection(s)
I20260809 02:18:28.803124 3753 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241741788589-15662-0/minicluster-data/master-1/data/info.pb
I20260809 02:18:28.805714 3857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:18:28.807484 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 3753
I20260809 02:18:28.807602 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241741788589-15662-0/minicluster-data/master-1/wal/instance
I20260809 02:18:28.807931 15662 external_mini_cluster.cc:1428] Setting key 848bb38c25337a4f58e4fda004a38f0c
I20260809 02:18:28.810585 3857 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.75.190" port: 40747 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.15.75.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.188@KRBTEST.COM" created.
Aug 09 02:18:28 dist-test-slave-s11t krb5kdc[3611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241908, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.189@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:18:28.824757 3747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:28.812394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44392 (local address 127.15.75.190:40747)
0809 02:18:28.812531 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:28.812535 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:28.813557 (+ 1022us) server_negotiation.cc:407] Connection header received
0809 02:18:28.814441 (+ 884us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:28.814445 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:28.814510 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:28.814612 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:28.815701 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:28.816245 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:28.817497 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:28.817743 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:28.817871 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:28.820572 (+ 2701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:28.820579 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:28.820585 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:18:28.820622 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:28.822187 (+ 1565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:28.822875 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:28.822878 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:28.822879 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:28.822950 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:28.823413 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:28.823416 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:28.823417 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:28.823606 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:28.823693 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:28.824457 (+ 764us) server_negotiation.cc:299] Negotiation successful
0809 02:18:28.824578 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 02:18:28.825866 3860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:28.812841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:40747 (local address 127.0.0.1:44392)
0809 02:18:28.813419 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:28.813458 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:28.814227 (+ 769us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:28.814775 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:28.814781 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:28.815098 (+ 317us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:18:28.815556 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:28.815568 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:28.816388 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:28.816391 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:28.817365 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:28.817375 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:28.817911 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:28.817916 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:28.818087 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:28.818660 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:18:28.818677 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:18:28.820366 (+ 1689us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:18:28.822323 (+ 1957us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:18:28.822330 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:18:28.822345 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:18:28.822755 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:18:28.823045 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:18:28.823052 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:18:28.823056 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:18:28.823268 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:18:28.823730 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:18:28.823740 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:18:28.824083 (+ 343us) client_negotiation.cc:769] Sending connection context
0809 02:18:28.824341 (+ 258us) client_negotiation.cc:240] Negotiation successful
0809 02:18:28.824742 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":153,"threads_started":1}
I20260809 02:18:28.827517 3857 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.75.189" port: 33671 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 02:18:28 dist-test-slave-s11t krb5kdc[3611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241908, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.189@KRBTEST.COM for kudu/127.15.75.189@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.188.keytab.
Entry for principal kudu/127.15.75.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.188.keytab.
I20260809 02:18:28.836994 3860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:28.828071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.189:33671 (local address 127.0.0.1:39174)
0809 02:18:28.828272 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:28.828290 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:28.828377 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:28.828922 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:28.828928 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:28.829104 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:18:28.829223 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:28.829229 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:28.830324 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:28.830327 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:28.831102 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:28.831117 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:28.831700 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:28.831703 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:28.831779 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:28.832171 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:18:28.832182 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:18:28.833574 (+ 1392us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:18:28.835456 (+ 1882us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:18:28.835461 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:18:28.835463 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:18:28.835651 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:18:28.835944 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:18:28.835949 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:18:28.835950 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:18:28.836052 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:18:28.836551 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:18:28.836553 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:18:28.836634 (+ 81us) client_negotiation.cc:769] Sending connection context
0809 02:18:28.836782 (+ 148us) client_negotiation.cc:240] Negotiation successful
0809 02:18:28.836891 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121}
I20260809 02:18:28.837029 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241741788589-15662-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241741788589-15662-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241741788589-15662-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241741788589-15662-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.188:34753
--webserver_interface=127.15.75.188
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.188.keytab
--principal=kudu/127.15.75.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:45851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.75.190:40747,127.15.75.189:33671,127.15.75.188:34753
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 02:18:28.837248 3864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:28.828214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39174 (local address 127.15.75.189:33671)
0809 02:18:28.828526 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:28.828536 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:28.828555 (+ 19us) server_negotiation.cc:407] Connection header received
0809 02:18:28.828590 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:28.828593 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:28.828781 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:28.828890 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:28.829339 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:28.830169 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:28.831226 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:28.831569 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:28.831652 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:28.833706 (+ 2054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:28.833719 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:28.833735 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:18:28.833766 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:28.835330 (+ 1564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:28.835761 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:28.835766 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:28.835768 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:28.835851 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:28.836150 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:28.836152 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:28.836154 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:28.836432 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:28.836572 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:28.836890 (+ 318us) server_negotiation.cc:299] Negotiation successful
0809 02:18:28.837041 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":136,"threads_started":1}
I20260809 02:18:28.838909 3857 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.75.188" port: 34753 } has no permanent_uuid. Determining permanent_uuid...
I20260809 02:18:28.839622 3860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:28.839319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.188:34753 (local address 127.0.0.1:45858)
0809 02:18:28.839450 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:28.839545 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.188:34753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260809 02:18:28.839854 3780 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.75.188:34753: connect: Connection refused (error 111)
I20260809 02:18:28.840350 3744 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.75.189" port: 33671 } attempt: 3
I20260809 02:18:28.840798 3860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:28.840554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.188:34753 (local address 127.0.0.1:45870)
0809 02:18:28.840668 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:28.840740 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.188:34753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260809 02:18:28.841171 3857 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.75.188:34753: Network error: Client connection negotiation failed: client connection to 127.15.75.188:34753: connect: Connection refused (error 111)
Aug 09 02:18:28 dist-test-slave-s11t krb5kdc[3611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241908, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.190@KRBTEST.COM for kudu/127.15.75.189@KRBTEST.COM
I20260809 02:18:28.849531 3864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:28.841214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39186 (local address 127.15.75.189:33671)
0809 02:18:28.841339 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:28.841343 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:28.841356 (+ 13us) server_negotiation.cc:407] Connection header received
0809 02:18:28.841394 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:28.841396 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:28.841428 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:28.841489 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:28.842059 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:28.842529 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:28.843762 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:28.843996 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:28.844076 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:28.846386 (+ 2310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:28.846389 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:28.846391 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:18:28.846412 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:28.847771 (+ 1359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:28.848305 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:28.848308 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:28.848310 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:28.848379 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:28.848740 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:28.848743 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:28.848744 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:28.848919 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:28.848995 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:28.849355 (+ 360us) server_negotiation.cc:299] Negotiation successful
0809 02:18:28.849441 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 02:18:28.849773 3748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:28.840750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.189:33671 (local address 127.0.0.1:39186)
0809 02:18:28.840910 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:28.840927 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:28.841012 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:28.841554 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:28.841560 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:28.841760 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:18:28.841936 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:28.841948 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:28.842640 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:28.842646 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:28.843625 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:28.843641 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:28.844137 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:28.844143 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:28.844301 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:28.844746 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:18:28.844766 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:18:28.846220 (+ 1454us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:18:28.847897 (+ 1677us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:18:28.847904 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:18:28.847908 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:18:28.848188 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:18:28.848477 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:18:28.848480 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:18:28.848482 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:18:28.848592 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:18:28.849027 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:18:28.849032 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:18:28.849160 (+ 128us) client_negotiation.cc:769] Sending connection context
0809 02:18:28.849301 (+ 141us) client_negotiation.cc:240] Negotiation successful
0809 02:18:28.849459 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260809 02:18:28.851042 3744 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.75.188" port: 34753 } has no permanent_uuid. Determining permanent_uuid...
I20260809 02:18:28.851687 3748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:28.851447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.188:34753 (local address 127.0.0.1:45876)
0809 02:18:28.851567 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:28.851623 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.188:34753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260809 02:18:28.852454 3748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:28.852194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.188:34753 (local address 127.0.0.1:45884)
0809 02:18:28.852346 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:28.852397 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.188:34753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
W20260809 02:18:28.852610 3744 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.75.188:34753: Network error: Client connection negotiation failed: client connection to 127.15.75.188:34753: connect: Connection refused (error 111)
I20260809 02:18:28.883774 3744 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.75.188" port: 34753 } attempt: 1
I20260809 02:18:28.884487 3748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:28.884201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.188:34753 (local address 127.0.0.1:45900)
0809 02:18:28.884345 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:28.884407 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.188:34753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260809 02:18:28.885655 3748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:28.885422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.188:34753 (local address 127.0.0.1:45906)
0809 02:18:28.885541 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:28.885595 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.188:34753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260809 02:18:28.885812 3744 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.75.188:34753: Network error: Client connection negotiation failed: client connection to 127.15.75.188:34753: connect: Connection refused (error 111)
I20260809 02:18:28.890408 3857 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.75.188" port: 34753 } attempt: 1
I20260809 02:18:28.891045 3860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:28.890812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.188:34753 (local address 127.0.0.1:45918)
0809 02:18:28.890936 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:28.890986 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.188:34753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260809 02:18:28.891700 3860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:28.891498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.188:34753 (local address 127.0.0.1:45924)
0809 02:18:28.891603 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:28.891653 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.188:34753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260809 02:18:28.891840 3857 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.75.188:34753: Network error: Client connection negotiation failed: client connection to 127.15.75.188:34753: connect: Connection refused (error 111)
W20260809 02:18:28.951953 3865 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:18:28.952209 3865 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:18:28.952282 3865 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:18:28.956177 3865 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:18:28.956259 3865 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:18:28.956293 3865 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:18:28.956333 3865 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:18:28.956388 3865 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:18:28.960009 3857 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.75.188" port: 34753 } attempt: 2
I20260809 02:18:28.960911 3860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:28.960524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.188:34753 (local address 127.0.0.1:45940)
0809 02:18:28.960677 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:28.960832 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.188:34753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260809 02:18:28.960996 3744 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.75.188" port: 34753 } attempt: 2
I20260809 02:18:28.961735 3748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:28.961444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.188:34753 (local address 127.0.0.1:45952)
0809 02:18:28.961591 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:28.961656 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.188:34753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260809 02:18:28.961755 3860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:28.961506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.188:34753 (local address 127.0.0.1:45958)
0809 02:18:28.961651 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:28.961701 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.188:34753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260809 02:18:28.961843 3865 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:45851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241741788589-15662-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241741788589-15662-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.15.75.190:40747,127.15.75.189:33671,127.15.75.188:34753
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.188.keytab
--principal=kudu/127.15.75.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.188:34753
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241741788589-15662-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.15.75.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3865
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241741788589-15662-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
W20260809 02:18:28.961921 3857 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.75.188:34753: Network error: Client connection negotiation failed: client connection to 127.15.75.188:34753: connect: Connection refused (error 111)
I20260809 02:18:28.962761 3876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:28.962213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.188:34753 (local address 127.0.0.1:45972)
0809 02:18:28.962592 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:28.962640 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.188:34753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":139,"threads_started":1}
W20260809 02:18:28.962922 3744 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.75.188:34753: Network error: Client connection negotiation failed: client connection to 127.15.75.188:34753: connect: Connection refused (error 111)
I20260809 02:18:28.963064 3865 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:18:28.964013 3865 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:18:28.972146 3878 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:28.972203 3882 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:28.972146 3879 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:28.972245 3865 server_base.cc:1034] running on GCE node
I20260809 02:18:28.973016 3865 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:18:28.974041 3865 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:18:28.975236 3865 hybrid_clock.cc:648] HybridClock initialized: now 1786241908975226 us; error 26 us; skew 500 ppm
Aug 09 02:18:28 dist-test-slave-s11t krb5kdc[3611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241908, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:18:28.978008 3865 init.cc:377] Logged in from keytab as kudu/127.15.75.188@KRBTEST.COM (short username kudu)
I20260809 02:18:28.978899 3865 webserver.cc:460] Webserver started at http://127.15.75.188:45113/ using document root <none> and password file <none>
I20260809 02:18:28.979444 3865 fs_manager.cc:359] Metadata directory not provided
I20260809 02:18:28.979537 3865 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:18:28.979772 3865 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:18:28.981778 3865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241741788589-15662-0/minicluster-data/master-2/data/instance:
uuid: "3b3456661ed643828c12b66cbcd48090"
format_stamp: "Formatted at 2026-08-09 02:18:28 on dist-test-slave-s11t"
server_key: "f3a38b0c474e70f381420c3c3b8f8aa4"
server_key_iv: "22697076272eb088fa08eea0e5e586e6"
server_key_version: "encryptionkey@0"
I20260809 02:18:28.982393 3865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241741788589-15662-0/minicluster-data/master-2/wal/instance:
uuid: "3b3456661ed643828c12b66cbcd48090"
format_stamp: "Formatted at 2026-08-09 02:18:28 on dist-test-slave-s11t"
server_key: "f3a38b0c474e70f381420c3c3b8f8aa4"
server_key_iv: "22697076272eb088fa08eea0e5e586e6"
server_key_version: "encryptionkey@0"
I20260809 02:18:28.986042 3865 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 02:18:28.988322 3896 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:28.989517 3865 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 02:18:28.989665 3865 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241741788589-15662-0/minicluster-data/master-2/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241741788589-15662-0/minicluster-data/master-2/wal
uuid: "3b3456661ed643828c12b66cbcd48090"
format_stamp: "Formatted at 2026-08-09 02:18:28 on dist-test-slave-s11t"
server_key: "f3a38b0c474e70f381420c3c3b8f8aa4"
server_key_iv: "22697076272eb088fa08eea0e5e586e6"
server_key_version: "encryptionkey@0"
I20260809 02:18:28.989773 3865 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241741788589-15662-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241741788589-15662-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241741788589-15662-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:18:29.050405 3865 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:18:29.053071 3857 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.75.188" port: 34753 } attempt: 3
I20260809 02:18:29.053447 3865 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:18:29.053658 3865 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:18:29.053905 3860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:29.053581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.188:34753 (local address 127.0.0.1:45988)
0809 02:18:29.053717 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:29.053801 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.188:34753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260809 02:18:29.054745 3860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:29.054449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.188:34753 (local address 127.0.0.1:46000)
0809 02:18:29.054621 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:29.054674 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.188:34753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
W20260809 02:18:29.054947 3857 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.75.188:34753: Network error: Client connection negotiation failed: client connection to 127.15.75.188:34753: connect: Connection refused (error 111)
I20260809 02:18:29.061004 3865 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.188:34753
I20260809 02:18:29.061029 3972 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.188:34753 every 8 connection(s)
I20260809 02:18:29.061956 3865 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241741788589-15662-0/minicluster-data/master-2/data/info.pb
I20260809 02:18:29.062098 3744 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.75.188" port: 34753 } attempt: 3
I20260809 02:18:29.064921 3974 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:18:29.064929 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 3865
I20260809 02:18:29.065057 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241741788589-15662-0/minicluster-data/master-2/wal/instance
I20260809 02:18:29.065363 15662 external_mini_cluster.cc:1428] Setting key d989a1266d645ad9ab68261611a5a08e
I20260809 02:18:29.070168 3974 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.75.190" port: 40747 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 02:18:29 dist-test-slave-s11t krb5kdc[3611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241908, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.190@KRBTEST.COM for kudu/127.15.75.188@KRBTEST.COM
Aug 09 02:18:29 dist-test-slave-s11t krb5kdc[3611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241908, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.75.188@KRBTEST.COM
I20260809 02:18:29.076040 3876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:29.062572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.188:34753 (local address 127.0.0.1:46014)
0809 02:18:29.062723 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:29.062740 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:29.062846 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:29.065016 (+ 2170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:29.065020 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:29.065228 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:18:29.065385 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:29.065392 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:29.067131 (+ 1739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:29.067135 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:29.067846 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:29.067856 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:29.068802 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:29.068808 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:29.068920 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:29.069469 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:18:29.069484 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:18:29.071092 (+ 1608us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:18:29.073915 (+ 2823us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:18:29.073918 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:18:29.073921 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:18:29.074191 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:18:29.074613 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:18:29.074619 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:18:29.074624 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:18:29.074746 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:18:29.075478 (+ 732us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:18:29.075481 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:18:29.075587 (+ 106us) client_negotiation.cc:769] Sending connection context
0809 02:18:29.075685 (+ 98us) client_negotiation.cc:240] Negotiation successful
0809 02:18:29.075829 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260809 02:18:29.076990 3976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:29.063198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46014 (local address 127.15.75.188:34753)
0809 02:18:29.063600 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:29.063618 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:29.063666 (+ 48us) server_negotiation.cc:407] Connection header received
0809 02:18:29.064291 (+ 625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:29.064306 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:29.064613 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:29.065071 (+ 458us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:29.065508 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:29.067011 (+ 1503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:29.067968 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:29.068578 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:29.068884 (+ 306us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:29.071305 (+ 2421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:29.071312 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:29.071328 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:18:29.071358 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:29.073799 (+ 2441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:29.074332 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:29.074340 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:29.074349 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:29.074495 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:29.074848 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:29.074851 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:29.074853 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:29.075355 (+ 502us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:29.075496 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:29.075847 (+ 351us) server_negotiation.cc:299] Negotiation successful
0809 02:18:29.076078 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":151,"threads_started":1}
I20260809 02:18:29.078114 3977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:29.065990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46020 (local address 127.15.75.188:34753)
0809 02:18:29.066233 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:29.066241 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:29.066499 (+ 258us) server_negotiation.cc:407] Connection header received
0809 02:18:29.066627 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:29.066630 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:29.066669 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:29.066757 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:29.067660 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:29.068464 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:29.069642 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:29.069899 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:29.070011 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:29.072730 (+ 2719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:29.072735 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:29.072740 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:18:29.072768 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:29.074554 (+ 1786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:29.075028 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:29.075031 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:29.075033 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:29.075107 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:29.075420 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:29.075425 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:29.075428 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:29.075620 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:29.075826 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:29.077112 (+ 1286us) server_negotiation.cc:299] Negotiation successful
0809 02:18:29.077939 (+ 827us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":95,"threads_started":1}
Aug 09 02:18:29 dist-test-slave-s11t krb5kdc[3611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241908, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.188@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:18:29.081637 3744 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e8cd179cc854127a506990625b56e38: Bootstrap starting.
I20260809 02:18:29.083734 3747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:29.072115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44396 (local address 127.15.75.190:40747)
0809 02:18:29.072261 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:29.072265 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:29.072936 (+ 671us) server_negotiation.cc:407] Connection header received
0809 02:18:29.073315 (+ 379us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:29.073321 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:29.073376 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:29.073464 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:29.074216 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:29.074760 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:29.075927 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:29.076237 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:29.076307 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:29.080172 (+ 3865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:29.080175 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:29.080178 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:18:29.080198 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:29.081684 (+ 1486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:29.082278 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:29.082283 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:29.082284 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:29.082364 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:29.082675 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:29.082691 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:29.082693 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:29.082954 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:29.083057 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:29.083478 (+ 421us) server_negotiation.cc:299] Negotiation successful
0809 02:18:29.083588 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 02:18:29.084198 3979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:29.072078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:40747 (local address 127.0.0.1:44396)
0809 02:18:29.072775 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:29.072835 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:29.072971 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:29.073530 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:29.073538 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:29.073999 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:18:29.074096 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:29.074101 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:29.074902 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:29.074905 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:29.075800 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:29.075811 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:29.076444 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:29.076446 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:29.076522 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:29.076998 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:18:29.077017 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:18:29.080035 (+ 3018us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:18:29.081839 (+ 1804us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:18:29.081848 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:18:29.081853 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:18:29.082170 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:18:29.082460 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:18:29.082463 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:18:29.082465 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:18:29.082551 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:18:29.083117 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:18:29.083123 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:18:29.083252 (+ 129us) client_negotiation.cc:769] Sending connection context
0809 02:18:29.083857 (+ 605us) client_negotiation.cc:240] Negotiation successful
0809 02:18:29.084037 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":98,"threads_started":1}
I20260809 02:18:29.084555 3744 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6e8cd179cc854127a506990625b56e38: Neither blocks nor log segments found. Creating new log.
I20260809 02:18:29.085361 3974 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.75.189" port: 33671 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 02:18:29 dist-test-slave-s11t krb5kdc[3611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241908, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:18:29.085669 3744 log.cc:824] T 00000000000000000000000000000000 P 6e8cd179cc854127a506990625b56e38: Log is configured to *not* fsync() on all Append() calls
I20260809 02:18:29.087728 3744 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e8cd179cc854127a506990625b56e38: No bootstrap required, opened a new log
I20260809 02:18:29.089557 3980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:29.079606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44402 (local address 127.15.75.190:40747)
0809 02:18:29.079803 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:29.079816 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:29.079831 (+ 15us) server_negotiation.cc:407] Connection header received
0809 02:18:29.079903 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:29.079906 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:29.079950 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:29.080046 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:29.080822 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:29.081562 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:29.082617 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:29.082898 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:29.083009 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:29.086048 (+ 3039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:29.086052 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:29.086054 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:18:29.086078 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:29.087661 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:29.088184 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:29.088187 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:29.088189 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:29.088276 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:29.088585 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:29.088589 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:29.088590 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:29.088826 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:29.088987 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:29.089249 (+ 262us) server_negotiation.cc:299] Negotiation successful
0809 02:18:29.089369 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":61,"threads_started":1}
Aug 09 02:18:29 dist-test-slave-s11t krb5kdc[3611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241908, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.188@KRBTEST.COM for kudu/127.15.75.189@KRBTEST.COM
I20260809 02:18:29.093043 3744 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6e8cd179cc854127a506990625b56e38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8cd179cc854127a506990625b56e38" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 40747 } } peers { permanent_uuid: "c70d1718ca4e42ec8b887716d3cccd56" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 33671 } } peers { permanent_uuid: "3b3456661ed643828c12b66cbcd48090" member_type: VOTER last_known_addr { host: "127.15.75.188" port: 34753 } }
I20260809 02:18:29.093273 3744 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6e8cd179cc854127a506990625b56e38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:18:29.093346 3744 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6e8cd179cc854127a506990625b56e38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e8cd179cc854127a506990625b56e38, State: Initialized, Role: FOLLOWER
I20260809 02:18:29.093814 3744 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e8cd179cc854127a506990625b56e38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8cd179cc854127a506990625b56e38" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 40747 } } peers { permanent_uuid: "c70d1718ca4e42ec8b887716d3cccd56" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 33671 } } peers { permanent_uuid: "3b3456661ed643828c12b66cbcd48090" member_type: VOTER last_known_addr { host: "127.15.75.188" port: 34753 } }
I20260809 02:18:29.095214 3744 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6e8cd179cc854127a506990625b56e38 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:18:29.095397 3979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:29.085832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.189:33671 (local address 127.0.0.1:39192)
0809 02:18:29.085965 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:29.085981 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:29.086058 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:29.086755 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:29.086760 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:29.086926 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:18:29.087063 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:29.087070 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:29.088022 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:29.088026 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:29.088975 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:29.088987 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:29.089647 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:29.089651 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:29.089744 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:29.090216 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:18:29.090230 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:18:29.091691 (+ 1461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:18:29.093708 (+ 2017us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:18:29.093711 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:18:29.093713 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:18:29.093964 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:18:29.094406 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:18:29.094409 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:18:29.094410 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:18:29.094498 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:18:29.094916 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:18:29.094919 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:18:29.095012 (+ 93us) client_negotiation.cc:769] Sending connection context
0809 02:18:29.095119 (+ 107us) client_negotiation.cc:240] Negotiation successful
0809 02:18:29.095251 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260809 02:18:29.095453 3864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:29.086077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39192 (local address 127.15.75.189:33671)
0809 02:18:29.086209 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:29.086214 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:29.086228 (+ 14us) server_negotiation.cc:407] Connection header received
0809 02:18:29.086277 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:29.086282 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:29.086605 (+ 323us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:29.086695 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:29.087183 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:29.087903 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:29.089088 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:29.089415 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:29.089524 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:29.091837 (+ 2313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:29.091840 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:29.091842 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:18:29.091874 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:29.093614 (+ 1740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:29.094200 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:29.094203 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:29.094205 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:29.094292 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:29.094602 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:29.094605 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:29.094607 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:29.094801 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:29.094914 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:29.095195 (+ 281us) server_negotiation.cc:299] Negotiation successful
0809 02:18:29.095314 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 02:18:29.096520 3974 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.75.188" port: 34753 } has no permanent_uuid. Determining permanent_uuid...
I20260809 02:18:29.097393 3983 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e8cd179cc854127a506990625b56e38 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8cd179cc854127a506990625b56e38" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 40747 } } peers { permanent_uuid: "c70d1718ca4e42ec8b887716d3cccd56" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 33671 } } peers { permanent_uuid: "3b3456661ed643828c12b66cbcd48090" member_type: VOTER last_known_addr { host: "127.15.75.188" port: 34753 } } }
I20260809 02:18:29.097546 3983 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e8cd179cc854127a506990625b56e38 [sys.catalog]: This master's current role is: FOLLOWER
Aug 09 02:18:29 dist-test-slave-s11t krb5kdc[3611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241908, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.75.189@KRBTEST.COM
W20260809 02:18:29.101096 3996 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6e8cd179cc854127a506990625b56e38: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:18:29.101189 3996 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:18:29.103787 3982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:29.090627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39200 (local address 127.15.75.189:33671)
0809 02:18:29.091995 (+ 1368us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:29.092004 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:29.092016 (+ 12us) server_negotiation.cc:407] Connection header received
0809 02:18:29.092048 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:29.092051 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:29.092098 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:29.092477 (+ 379us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:29.093323 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:29.094072 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:29.094791 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:29.095163 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:29.095256 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:29.098509 (+ 3253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:29.098516 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:29.098522 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:18:29.098546 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:29.101566 (+ 3020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:29.102051 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:29.102054 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:29.102056 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:29.102141 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:29.102447 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:29.102451 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:29.102452 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:29.102663 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:29.102858 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:29.103490 (+ 632us) server_negotiation.cc:299] Negotiation successful
0809 02:18:29.103609 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1297,"thread_start_us":103,"threads_started":1}
I20260809 02:18:29.100776 3637 master_runner.cc:186] Error getting master registration for 127.15.75.189:33671: OK, instance_id { permanent_uuid: "c70d1718ca4e42ec8b887716d3cccd56" instance_seqno: 1786241908800172 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 02:18:29.104382 3637 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 09 02:18:29 dist-test-slave-s11t krb5kdc[3611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241908, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.188@KRBTEST.COM for kudu/127.15.75.188@KRBTEST.COM
I20260809 02:18:29.106868 15662 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:18:29.109771 3979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:29.097031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.188:34753 (local address 127.0.0.1:46026)
0809 02:18:29.097236 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:29.097249 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:29.097335 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:29.097729 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:29.097732 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:29.097908 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:18:29.098009 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:29.098020 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:29.099041 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:29.099044 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:29.099789 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:29.099800 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:29.100445 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:29.100449 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:29.100550 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:29.104508 (+ 3958us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:18:29.104523 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:18:29.105986 (+ 1463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:18:29.108023 (+ 2037us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:18:29.108026 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:18:29.108028 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:18:29.108256 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:18:29.108622 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:18:29.108625 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:18:29.108627 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:18:29.108754 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:18:29.109281 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:18:29.109287 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:18:29.109374 (+ 87us) client_negotiation.cc:769] Sending connection context
0809 02:18:29.109495 (+ 121us) client_negotiation.cc:240] Negotiation successful
0809 02:18:29.109627 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141}
I20260809 02:18:29.110453 3977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:29.097075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46026 (local address 127.15.75.188:34753)
0809 02:18:29.097198 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:29.097202 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:29.097417 (+ 215us) server_negotiation.cc:407] Connection header received
0809 02:18:29.097443 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:29.097445 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:29.097478 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:29.097538 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:29.098144 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:29.098902 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:29.099923 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:29.100259 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:29.100370 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:29.106128 (+ 5758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:29.106132 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:29.106134 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:18:29.106155 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:29.107909 (+ 1754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:29.108365 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:29.108370 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:29.108371 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:29.108495 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:29.108861 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:29.108864 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:29.108866 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:29.109140 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:29.109238 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:29.110170 (+ 932us) server_negotiation.cc:299] Negotiation successful
0809 02:18:29.110287 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 02:18:29.113197 3974 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b3456661ed643828c12b66cbcd48090: Bootstrap starting.
I20260809 02:18:29.115521 3974 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3b3456661ed643828c12b66cbcd48090: Neither blocks nor log segments found. Creating new log.
I20260809 02:18:29.116465 3974 log.cc:824] T 00000000000000000000000000000000 P 3b3456661ed643828c12b66cbcd48090: Log is configured to *not* fsync() on all Append() calls
I20260809 02:18:29.120772 3976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:29.109509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46042 (local address 127.15.75.188:34753)
0809 02:18:29.109617 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:29.109621 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:29.109765 (+ 144us) server_negotiation.cc:407] Connection header received
0809 02:18:29.109941 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:29.109945 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:29.109984 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:29.110033 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:29.111000 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:29.111770 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:29.115089 (+ 3319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:29.115438 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:29.115511 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:29.117331 (+ 1820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:29.117335 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:29.117338 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:18:29.117362 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:29.118947 (+ 1585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:29.119440 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:29.119444 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:29.119445 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:29.119524 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:29.119809 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:29.119813 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:29.119814 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:29.120004 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:29.120134 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:29.120486 (+ 352us) server_negotiation.cc:299] Negotiation successful
0809 02:18:29.120597 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 02:18:29.122028 3982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:29.109163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39214 (local address 127.15.75.189:33671)
0809 02:18:29.109265 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:29.109268 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:29.109843 (+ 575us) server_negotiation.cc:407] Connection header received
0809 02:18:29.110049 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:29.110054 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:29.110100 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:29.110155 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:29.113076 (+ 2921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:29.113852 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:29.114985 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:29.115310 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:29.115437 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:29.117510 (+ 2073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:29.117514 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:29.117517 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:18:29.117540 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:29.119669 (+ 2129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:29.120129 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:29.120132 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:29.120134 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:29.120213 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:29.120509 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:29.120512 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:29.120513 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:29.120755 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:29.121064 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:29.121441 (+ 377us) server_negotiation.cc:299] Negotiation successful
0809 02:18:29.121858 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 02:18:29.126034 3980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:29.113119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44408 (local address 127.15.75.190:40747)
0809 02:18:29.116813 (+ 3694us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:29.116819 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:29.117334 (+ 515us) server_negotiation.cc:407] Connection header received
0809 02:18:29.117511 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:29.117514 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:29.117557 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:29.117615 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:29.118522 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:29.119376 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:29.120510 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:29.120927 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:29.121001 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:29.122767 (+ 1766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:29.122771 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:29.122773 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:18:29.122801 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:29.124343 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:29.124774 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:29.124779 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:29.124781 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:29.124887 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:29.125161 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:29.125165 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:29.125167 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:29.125384 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:29.125525 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:29.125773 (+ 248us) server_negotiation.cc:299] Negotiation successful
0809 02:18:29.125886 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3598}
I20260809 02:18:29.126750 3974 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b3456661ed643828c12b66cbcd48090: No bootstrap required, opened a new log
I20260809 02:18:29.130271 3974 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3b3456661ed643828c12b66cbcd48090 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8cd179cc854127a506990625b56e38" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 40747 } } peers { permanent_uuid: "c70d1718ca4e42ec8b887716d3cccd56" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 33671 } } peers { permanent_uuid: "3b3456661ed643828c12b66cbcd48090" member_type: VOTER last_known_addr { host: "127.15.75.188" port: 34753 } }
I20260809 02:18:29.130527 3974 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3b3456661ed643828c12b66cbcd48090 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:18:29.130610 3974 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3b3456661ed643828c12b66cbcd48090 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b3456661ed643828c12b66cbcd48090, State: Initialized, Role: FOLLOWER
I20260809 02:18:29.131083 3974 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3b3456661ed643828c12b66cbcd48090 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8cd179cc854127a506990625b56e38" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 40747 } } peers { permanent_uuid: "c70d1718ca4e42ec8b887716d3cccd56" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 33671 } } peers { permanent_uuid: "3b3456661ed643828c12b66cbcd48090" member_type: VOTER last_known_addr { host: "127.15.75.188" port: 34753 } }
I20260809 02:18:29.131549 4011 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b3456661ed643828c12b66cbcd48090 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8cd179cc854127a506990625b56e38" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 40747 } } peers { permanent_uuid: "c70d1718ca4e42ec8b887716d3cccd56" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 33671 } } peers { permanent_uuid: "3b3456661ed643828c12b66cbcd48090" member_type: VOTER last_known_addr { host: "127.15.75.188" port: 34753 } } }
I20260809 02:18:29.131709 4011 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b3456661ed643828c12b66cbcd48090 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:18:29.132304 3974 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3b3456661ed643828c12b66cbcd48090 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 02:18:29.137647 4031 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3b3456661ed643828c12b66cbcd48090: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:18:29.137732 4031 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:18:29.139219 3865 master_runner.cc:186] Error getting master registration for 127.15.75.189:33671: OK, instance_id { permanent_uuid: "c70d1718ca4e42ec8b887716d3cccd56" instance_seqno: 1786241908800172 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 02:18:29.142413 3865 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 02:18:29.198114 3857 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.75.188" port: 34753 } attempt: 4
Aug 09 02:18:29 dist-test-slave-s11t krb5kdc[3611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241908, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.189@KRBTEST.COM for kudu/127.15.75.188@KRBTEST.COM
I20260809 02:18:29.208985 3860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:29.198529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.188:34753 (local address 127.0.0.1:46056)
0809 02:18:29.198655 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:29.198671 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:29.198757 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:29.199047 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:29.199051 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:29.199309 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:18:29.199423 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:29.199430 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:29.200444 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:29.200447 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:29.201329 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:29.201341 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:29.201926 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:29.201930 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:29.202025 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:29.202599 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:18:29.202614 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:18:29.204238 (+ 1624us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:18:29.207184 (+ 2946us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:18:29.207188 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:18:29.207190 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:18:29.207443 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:18:29.207742 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:18:29.207745 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:18:29.207747 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:18:29.207843 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:18:29.208367 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:18:29.208370 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:18:29.208494 (+ 124us) client_negotiation.cc:769] Sending connection context
0809 02:18:29.208604 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 02:18:29.208810 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260809 02:18:29.208997 3976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:29.198601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46056 (local address 127.15.75.188:34753)
0809 02:18:29.198713 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:29.198717 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:29.198732 (+ 15us) server_negotiation.cc:407] Connection header received
0809 02:18:29.198852 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:29.198858 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:29.198904 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:29.198987 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:29.199558 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:29.200309 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:29.201465 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:29.201798 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:29.201911 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:29.204417 (+ 2506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:29.204421 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:29.204425 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:18:29.204455 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:29.207059 (+ 2604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:29.207547 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:29.207551 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:29.207552 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:29.207643 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:29.207954 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:29.207959 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:29.207962 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:29.208206 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:29.208302 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:29.208716 (+ 414us) server_negotiation.cc:299] Negotiation successful
0809 02:18:29.208833 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 02:18:29.212363 3857 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c70d1718ca4e42ec8b887716d3cccd56: Bootstrap starting.
I20260809 02:18:29.215409 3857 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c70d1718ca4e42ec8b887716d3cccd56: Neither blocks nor log segments found. Creating new log.
I20260809 02:18:29.216382 3857 log.cc:824] T 00000000000000000000000000000000 P c70d1718ca4e42ec8b887716d3cccd56: Log is configured to *not* fsync() on all Append() calls
I20260809 02:18:29.219686 3857 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c70d1718ca4e42ec8b887716d3cccd56: No bootstrap required, opened a new log
I20260809 02:18:29.223876 3857 raft_consensus.cc:348] T 00000000000000000000000000000000 P c70d1718ca4e42ec8b887716d3cccd56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8cd179cc854127a506990625b56e38" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 40747 } } peers { permanent_uuid: "c70d1718ca4e42ec8b887716d3cccd56" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 33671 } } peers { permanent_uuid: "3b3456661ed643828c12b66cbcd48090" member_type: VOTER last_known_addr { host: "127.15.75.188" port: 34753 } }
I20260809 02:18:29.224094 3857 raft_consensus.cc:374] T 00000000000000000000000000000000 P c70d1718ca4e42ec8b887716d3cccd56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:18:29.224146 3857 raft_consensus.cc:729] T 00000000000000000000000000000000 P c70d1718ca4e42ec8b887716d3cccd56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c70d1718ca4e42ec8b887716d3cccd56, State: Initialized, Role: FOLLOWER
I20260809 02:18:29.224601 3857 consensus_queue.cc:260] T 00000000000000000000000000000000 P c70d1718ca4e42ec8b887716d3cccd56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8cd179cc854127a506990625b56e38" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 40747 } } peers { permanent_uuid: "c70d1718ca4e42ec8b887716d3cccd56" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 33671 } } peers { permanent_uuid: "3b3456661ed643828c12b66cbcd48090" member_type: VOTER last_known_addr { host: "127.15.75.188" port: 34753 } }
I20260809 02:18:29.225178 4036 sys_catalog.cc:455] T 00000000000000000000000000000000 P c70d1718ca4e42ec8b887716d3cccd56 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8cd179cc854127a506990625b56e38" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 40747 } } peers { permanent_uuid: "c70d1718ca4e42ec8b887716d3cccd56" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 33671 } } peers { permanent_uuid: "3b3456661ed643828c12b66cbcd48090" member_type: VOTER last_known_addr { host: "127.15.75.188" port: 34753 } } }
I20260809 02:18:29.225965 3857 sys_catalog.cc:564] T 00000000000000000000000000000000 P c70d1718ca4e42ec8b887716d3cccd56 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:18:29.226087 4036 sys_catalog.cc:458] T 00000000000000000000000000000000 P c70d1718ca4e42ec8b887716d3cccd56 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 02:18:29.231230 4054 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c70d1718ca4e42ec8b887716d3cccd56: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:18:29.231354 4054 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:18:29.233738 3753 master_runner.cc:263] No leader master found from master c70d1718ca4e42ec8b887716d3cccd56
I20260809 02:18:29.233812 3753 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260809 02:18:29.296631 3789 tablet.cc:2369] T 00000000000000000000000000000000 P c70d1718ca4e42ec8b887716d3cccd56: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 02:18:29.305336 3903 tablet.cc:2369] T 00000000000000000000000000000000 P 3b3456661ed643828c12b66cbcd48090: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 02:18:29.337914 3669 tablet.cc:2369] T 00000000000000000000000000000000 P 6e8cd179cc854127a506990625b56e38: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:18:29.342562 3983 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6e8cd179cc854127a506990625b56e38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:18:29.342767 3983 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6e8cd179cc854127a506990625b56e38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8cd179cc854127a506990625b56e38" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 40747 } } peers { permanent_uuid: "c70d1718ca4e42ec8b887716d3cccd56" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 33671 } } peers { permanent_uuid: "3b3456661ed643828c12b66cbcd48090" member_type: VOTER last_known_addr { host: "127.15.75.188" port: 34753 } }
I20260809 02:18:29.343859 3983 leader_election.cc:288] T 00000000000000000000000000000000 P 6e8cd179cc854127a506990625b56e38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c70d1718ca4e42ec8b887716d3cccd56 (127.15.75.189:33671), 3b3456661ed643828c12b66cbcd48090 (127.15.75.188:34753)
I20260809 02:18:29.344403 3818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6e8cd179cc854127a506990625b56e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c70d1718ca4e42ec8b887716d3cccd56" is_pre_election: true
I20260809 02:18:29.344393 3931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6e8cd179cc854127a506990625b56e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b3456661ed643828c12b66cbcd48090" is_pre_election: true
I20260809 02:18:29.344734 3818 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c70d1718ca4e42ec8b887716d3cccd56 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e8cd179cc854127a506990625b56e38 in term 0.
I20260809 02:18:29.344753 3931 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3b3456661ed643828c12b66cbcd48090 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e8cd179cc854127a506990625b56e38 in term 0.
I20260809 02:18:29.345183 3666 leader_election.cc:302] T 00000000000000000000000000000000 P 6e8cd179cc854127a506990625b56e38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e8cd179cc854127a506990625b56e38, c70d1718ca4e42ec8b887716d3cccd56; no voters:
I20260809 02:18:29.345482 3983 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6e8cd179cc854127a506990625b56e38 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:18:29.345572 3983 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6e8cd179cc854127a506990625b56e38 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:18:29.345616 3983 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6e8cd179cc854127a506990625b56e38 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:18:29.347172 3983 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6e8cd179cc854127a506990625b56e38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8cd179cc854127a506990625b56e38" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 40747 } } peers { permanent_uuid: "c70d1718ca4e42ec8b887716d3cccd56" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 33671 } } peers { permanent_uuid: "3b3456661ed643828c12b66cbcd48090" member_type: VOTER last_known_addr { host: "127.15.75.188" port: 34753 } }
I20260809 02:18:29.347554 3983 leader_election.cc:288] T 00000000000000000000000000000000 P 6e8cd179cc854127a506990625b56e38 [CANDIDATE]: Term 1 election: Requested vote from peers c70d1718ca4e42ec8b887716d3cccd56 (127.15.75.189:33671), 3b3456661ed643828c12b66cbcd48090 (127.15.75.188:34753)
I20260809 02:18:29.347872 3818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6e8cd179cc854127a506990625b56e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c70d1718ca4e42ec8b887716d3cccd56"
I20260809 02:18:29.347930 3931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6e8cd179cc854127a506990625b56e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b3456661ed643828c12b66cbcd48090"
I20260809 02:18:29.347987 3818 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c70d1718ca4e42ec8b887716d3cccd56 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:18:29.348029 3931 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3b3456661ed643828c12b66cbcd48090 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:18:29.349375 3931 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3b3456661ed643828c12b66cbcd48090 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e8cd179cc854127a506990625b56e38 in term 1.
I20260809 02:18:29.349681 3661 leader_election.cc:302] T 00000000000000000000000000000000 P 6e8cd179cc854127a506990625b56e38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b3456661ed643828c12b66cbcd48090, 6e8cd179cc854127a506990625b56e38; no voters:
I20260809 02:18:29.349869 3818 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c70d1718ca4e42ec8b887716d3cccd56 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e8cd179cc854127a506990625b56e38 in term 1.
I20260809 02:18:29.349891 3983 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6e8cd179cc854127a506990625b56e38 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:18:29.350129 3983 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6e8cd179cc854127a506990625b56e38 [term 1 LEADER]: Becoming Leader. State: Replica: 6e8cd179cc854127a506990625b56e38, State: Running, Role: LEADER
I20260809 02:18:29.350458 3983 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6e8cd179cc854127a506990625b56e38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8cd179cc854127a506990625b56e38" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 40747 } } peers { permanent_uuid: "c70d1718ca4e42ec8b887716d3cccd56" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 33671 } } peers { permanent_uuid: "3b3456661ed643828c12b66cbcd48090" member_type: VOTER last_known_addr { host: "127.15.75.188" port: 34753 } }
I20260809 02:18:29.351883 3983 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e8cd179cc854127a506990625b56e38 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e8cd179cc854127a506990625b56e38. Latest consensus state: current_term: 1 leader_uuid: "6e8cd179cc854127a506990625b56e38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8cd179cc854127a506990625b56e38" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 40747 } } peers { permanent_uuid: "c70d1718ca4e42ec8b887716d3cccd56" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 33671 } } peers { permanent_uuid: "3b3456661ed643828c12b66cbcd48090" member_type: VOTER last_known_addr { host: "127.15.75.188" port: 34753 } } }
I20260809 02:18:29.351989 3983 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e8cd179cc854127a506990625b56e38 [sys.catalog]: This master's current role is: LEADER
I20260809 02:18:29.352388 4063 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:18:29.353525 4063 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:18:29.358137 3818 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c70d1718ca4e42ec8b887716d3cccd56 [term 1 FOLLOWER]: Refusing update from remote peer 6e8cd179cc854127a506990625b56e38: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:18:29.358186 3931 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3b3456661ed643828c12b66cbcd48090 [term 1 FOLLOWER]: Refusing update from remote peer 6e8cd179cc854127a506990625b56e38: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:18:29.358783 3983 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6e8cd179cc854127a506990625b56e38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b3456661ed643828c12b66cbcd48090" member_type: VOTER last_known_addr { host: "127.15.75.188" port: 34753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:18:29.358958 4058 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6e8cd179cc854127a506990625b56e38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c70d1718ca4e42ec8b887716d3cccd56" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 33671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:18:29.362840 4036 sys_catalog.cc:455] T 00000000000000000000000000000000 P c70d1718ca4e42ec8b887716d3cccd56 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e8cd179cc854127a506990625b56e38. Latest consensus state: current_term: 1 leader_uuid: "6e8cd179cc854127a506990625b56e38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8cd179cc854127a506990625b56e38" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 40747 } } peers { permanent_uuid: "c70d1718ca4e42ec8b887716d3cccd56" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 33671 } } peers { permanent_uuid: "3b3456661ed643828c12b66cbcd48090" member_type: VOTER last_known_addr { host: "127.15.75.188" port: 34753 } } }
I20260809 02:18:29.362960 4036 sys_catalog.cc:458] T 00000000000000000000000000000000 P c70d1718ca4e42ec8b887716d3cccd56 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:18:29.362908 4011 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b3456661ed643828c12b66cbcd48090 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e8cd179cc854127a506990625b56e38. Latest consensus state: current_term: 1 leader_uuid: "6e8cd179cc854127a506990625b56e38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8cd179cc854127a506990625b56e38" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 40747 } } peers { permanent_uuid: "c70d1718ca4e42ec8b887716d3cccd56" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 33671 } } peers { permanent_uuid: "3b3456661ed643828c12b66cbcd48090" member_type: VOTER last_known_addr { host: "127.15.75.188" port: 34753 } } }
I20260809 02:18:29.363019 4011 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b3456661ed643828c12b66cbcd48090 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:18:29.365474 3983 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e8cd179cc854127a506990625b56e38 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6e8cd179cc854127a506990625b56e38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8cd179cc854127a506990625b56e38" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 40747 } } peers { permanent_uuid: "c70d1718ca4e42ec8b887716d3cccd56" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 33671 } } peers { permanent_uuid: "3b3456661ed643828c12b66cbcd48090" member_type: VOTER last_known_addr { host: "127.15.75.188" port: 34753 } } }
I20260809 02:18:29.365576 3983 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e8cd179cc854127a506990625b56e38 [sys.catalog]: This master's current role is: LEADER
I20260809 02:18:29.366664 4011 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b3456661ed643828c12b66cbcd48090 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6e8cd179cc854127a506990625b56e38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8cd179cc854127a506990625b56e38" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 40747 } } peers { permanent_uuid: "c70d1718ca4e42ec8b887716d3cccd56" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 33671 } } peers { permanent_uuid: "3b3456661ed643828c12b66cbcd48090" member_type: VOTER last_known_addr { host: "127.15.75.188" port: 34753 } } }
I20260809 02:18:29.366765 4011 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b3456661ed643828c12b66cbcd48090 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:18:29.366967 4063 catalog_manager.cc:1347] Generated new cluster ID: eb6f203da4824a3588f464c113930e42
I20260809 02:18:29.367026 4063 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:18:29.367447 4070 mvcc.cc:205] Tried to move back new op lower bound from 7316446860725010432 to 7316446860701806592. Current Snapshot: MvccSnapshot[applied={T|T < 7316446860725010432}]
I20260809 02:18:29.367446 4058 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e8cd179cc854127a506990625b56e38 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6e8cd179cc854127a506990625b56e38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8cd179cc854127a506990625b56e38" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 40747 } } peers { permanent_uuid: "c70d1718ca4e42ec8b887716d3cccd56" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 33671 } } peers { permanent_uuid: "3b3456661ed643828c12b66cbcd48090" member_type: VOTER last_known_addr { host: "127.15.75.188" port: 34753 } } }
I20260809 02:18:29.367548 4058 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e8cd179cc854127a506990625b56e38 [sys.catalog]: This master's current role is: LEADER
I20260809 02:18:29.373562 4067 mvcc.cc:205] Tried to move back new op lower bound from 7316446860725010432 to 7316446860701806592. Current Snapshot: MvccSnapshot[applied={T|T < 7316446860725010432}]
I20260809 02:18:29.373988 4036 sys_catalog.cc:455] T 00000000000000000000000000000000 P c70d1718ca4e42ec8b887716d3cccd56 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6e8cd179cc854127a506990625b56e38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8cd179cc854127a506990625b56e38" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 40747 } } peers { permanent_uuid: "c70d1718ca4e42ec8b887716d3cccd56" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 33671 } } peers { permanent_uuid: "3b3456661ed643828c12b66cbcd48090" member_type: VOTER last_known_addr { host: "127.15.75.188" port: 34753 } } }
I20260809 02:18:29.374087 4036 sys_catalog.cc:458] T 00000000000000000000000000000000 P c70d1718ca4e42ec8b887716d3cccd56 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:18:29.377470 4063 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:18:29.378665 4063 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:18:29.384582 4063 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6e8cd179cc854127a506990625b56e38: Generated new TSK 0
I20260809 02:18:29.385375 4063 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:18:30.139014 4031 catalog_manager.cc:1259] Loaded cluster ID: eb6f203da4824a3588f464c113930e42
I20260809 02:18:30.139127 4031 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3b3456661ed643828c12b66cbcd48090: loading cluster ID for follower catalog manager: success
I20260809 02:18:30.140882 4031 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3b3456661ed643828c12b66cbcd48090: acquiring CA information for follower catalog manager: success
I20260809 02:18:30.142045 4031 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3b3456661ed643828c12b66cbcd48090: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 02:18:30.232810 4054 catalog_manager.cc:1259] Loaded cluster ID: eb6f203da4824a3588f464c113930e42
I20260809 02:18:30.232887 4054 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c70d1718ca4e42ec8b887716d3cccd56: loading cluster ID for follower catalog manager: success
I20260809 02:18:30.234563 4054 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c70d1718ca4e42ec8b887716d3cccd56: acquiring CA information for follower catalog manager: success
I20260809 02:18:30.235702 4054 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c70d1718ca4e42ec8b887716d3cccd56: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 02:18:30.448575 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 3637
I20260809 02:18:30.463052 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 3753
I20260809 02:18:30.474820 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 3865
2026-08-09T02:18:30Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3427 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241741788589-15662-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:18:30 dist-test-slave-s11t krb5kdc[4086](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:18:30 dist-test-slave-s11t krb5kdc[4086](info): set up 2 sockets
Aug 09 02:18:30 dist-test-slave-s11t krb5kdc[4086](info): commencing operation
krb5kdc: starting...
W20260809 02:18:31.719877 15662 mini_kdc.cc:121] Time spent starting KDC: real 1.218s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Aug 09 02:18:31 dist-test-slave-s11t krb5kdc[4086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241911, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:18:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:18:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.75.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab.
Entry for principal kudu/127.15.75.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab.
I20260809 02:18:31.841392 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241741788589-15662-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241741788589-15662-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241741788589-15662-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:46835
--webserver_interface=127.15.75.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab
--principal=kudu/127.15.75.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:40675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.75.190:46835
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:18:31.952049 4109 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:18:31.952307 4109 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:18:31.952346 4109 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:18:31.956235 4109 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:18:31.956285 4109 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:18:31.956305 4109 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:18:31.956321 4109 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:18:31.956336 4109 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:18:31.961354 4109 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:40675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241741788589-15662-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241741788589-15662-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.75.190:46835
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab
--principal=kudu/127.15.75.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:46835
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.75.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4109
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241741788589-15662-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:18:31.962244 4109 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:18:31.963171 4109 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:18:31.971653 4119 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:31.971675 4121 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:31.971662 4117 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:31.971939 4109 server_base.cc:1034] running on GCE node
I20260809 02:18:31.972954 4109 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:18:31.974014 4109 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:18:31.975188 4109 hybrid_clock.cc:648] HybridClock initialized: now 1786241911975156 us; error 44 us; skew 500 ppm
Aug 09 02:18:31 dist-test-slave-s11t krb5kdc[4086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241911, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:18:31.978132 4109 init.cc:377] Logged in from keytab as kudu/127.15.75.190@KRBTEST.COM (short username kudu)
I20260809 02:18:31.979071 4109 webserver.cc:460] Webserver started at http://127.15.75.190:35781/ using document root <none> and password file <none>
I20260809 02:18:31.979645 4109 fs_manager.cc:359] Metadata directory not provided
I20260809 02:18:31.979748 4109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:18:31.979991 4109 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:18:31.982172 4109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241741788589-15662-0/minicluster-data/master-0/data/instance:
uuid: "36f1f9a0d13f487ebeabdbbf1a07f259"
format_stamp: "Formatted at 2026-08-09 02:18:31 on dist-test-slave-s11t"
server_key: "9cb2c33c666c8f7be0418de3efd2f16f"
server_key_iv: "ccd5a94e07d70d2fda514bb395318dbd"
server_key_version: "encryptionkey@0"
I20260809 02:18:31.982782 4109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241741788589-15662-0/minicluster-data/master-0/wal/instance:
uuid: "36f1f9a0d13f487ebeabdbbf1a07f259"
format_stamp: "Formatted at 2026-08-09 02:18:31 on dist-test-slave-s11t"
server_key: "9cb2c33c666c8f7be0418de3efd2f16f"
server_key_iv: "ccd5a94e07d70d2fda514bb395318dbd"
server_key_version: "encryptionkey@0"
I20260809 02:18:31.986485 4109 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 02:18:31.988905 4136 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:31.990091 4109 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 02:18:31.990247 4109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241741788589-15662-0/minicluster-data/master-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241741788589-15662-0/minicluster-data/master-0/wal
uuid: "36f1f9a0d13f487ebeabdbbf1a07f259"
format_stamp: "Formatted at 2026-08-09 02:18:31 on dist-test-slave-s11t"
server_key: "9cb2c33c666c8f7be0418de3efd2f16f"
server_key_iv: "ccd5a94e07d70d2fda514bb395318dbd"
server_key_version: "encryptionkey@0"
I20260809 02:18:31.990358 4109 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241741788589-15662-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241741788589-15662-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241741788589-15662-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:18:32.013077 4109 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:18:32.016166 4109 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:18:32.016374 4109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:18:32.024044 4109 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.190:46835
I20260809 02:18:32.024071 4215 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.190:46835 every 8 connection(s)
I20260809 02:18:32.025056 4109 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
I20260809 02:18:32.027521 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 4109
I20260809 02:18:32.027660 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241741788589-15662-0/minicluster-data/master-0/wal/instance
I20260809 02:18:32.027899 4216 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:18:32.027998 15662 external_mini_cluster.cc:1428] Setting key b698e9164c46a551ca6ba7c9c5f8db45
I20260809 02:18:32.035357 4216 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36f1f9a0d13f487ebeabdbbf1a07f259: Bootstrap starting.
Aug 09 02:18:32 dist-test-slave-s11t krb5kdc[4086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241911, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:18:32.037871 4216 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 36f1f9a0d13f487ebeabdbbf1a07f259: Neither blocks nor log segments found. Creating new log.
I20260809 02:18:32.038710 4216 log.cc:824] T 00000000000000000000000000000000 P 36f1f9a0d13f487ebeabdbbf1a07f259: Log is configured to *not* fsync() on all Append() calls
I20260809 02:18:32.040820 4216 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36f1f9a0d13f487ebeabdbbf1a07f259: No bootstrap required, opened a new log
I20260809 02:18:32.043036 4219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:32.029405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60822 (local address 127.15.75.190:46835)
0809 02:18:32.029859 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:32.029879 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:32.029921 (+ 42us) server_negotiation.cc:407] Connection header received
0809 02:18:32.030798 (+ 877us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:32.030824 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:32.031247 (+ 423us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:32.031612 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:32.032268 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:32.033795 (+ 1527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:32.034663 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:32.035342 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:32.035514 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:32.038348 (+ 2834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:32.038353 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:32.038365 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:18:32.038391 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:32.040219 (+ 1828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:32.040776 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:32.040781 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:32.040785 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:32.040889 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:32.041221 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:32.041224 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:32.041225 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:32.041590 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:32.041733 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:32.042045 (+ 312us) server_negotiation.cc:299] Negotiation successful
0809 02:18:32.042335 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":122,"threads_started":1}
I20260809 02:18:32.044625 4216 raft_consensus.cc:348] T 00000000000000000000000000000000 P 36f1f9a0d13f487ebeabdbbf1a07f259 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36f1f9a0d13f487ebeabdbbf1a07f259" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 46835 } }
I20260809 02:18:32.045006 4216 raft_consensus.cc:374] T 00000000000000000000000000000000 P 36f1f9a0d13f487ebeabdbbf1a07f259 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:18:32.045079 4216 raft_consensus.cc:729] T 00000000000000000000000000000000 P 36f1f9a0d13f487ebeabdbbf1a07f259 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36f1f9a0d13f487ebeabdbbf1a07f259, State: Initialized, Role: FOLLOWER
I20260809 02:18:32.045557 4216 consensus_queue.cc:260] T 00000000000000000000000000000000 P 36f1f9a0d13f487ebeabdbbf1a07f259 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36f1f9a0d13f487ebeabdbbf1a07f259" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 46835 } }
I20260809 02:18:32.045733 4216 raft_consensus.cc:388] T 00000000000000000000000000000000 P 36f1f9a0d13f487ebeabdbbf1a07f259 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:18:32.045799 15662 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:18:32.045814 4216 raft_consensus.cc:482] T 00000000000000000000000000000000 P 36f1f9a0d13f487ebeabdbbf1a07f259 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:18:32.045943 4216 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 36f1f9a0d13f487ebeabdbbf1a07f259 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:18:32.046844 4216 raft_consensus.cc:504] T 00000000000000000000000000000000 P 36f1f9a0d13f487ebeabdbbf1a07f259 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36f1f9a0d13f487ebeabdbbf1a07f259" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 46835 } }
I20260809 02:18:32.047195 4216 leader_election.cc:302] T 00000000000000000000000000000000 P 36f1f9a0d13f487ebeabdbbf1a07f259 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 36f1f9a0d13f487ebeabdbbf1a07f259; no voters:
I20260809 02:18:32.047542 4216 leader_election.cc:288] T 00000000000000000000000000000000 P 36f1f9a0d13f487ebeabdbbf1a07f259 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:18:32.047654 4221 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 36f1f9a0d13f487ebeabdbbf1a07f259 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:18:32.047911 4221 raft_consensus.cc:686] T 00000000000000000000000000000000 P 36f1f9a0d13f487ebeabdbbf1a07f259 [term 1 LEADER]: Becoming Leader. State: Replica: 36f1f9a0d13f487ebeabdbbf1a07f259, State: Running, Role: LEADER
I20260809 02:18:32.048246 4221 consensus_queue.cc:237] T 00000000000000000000000000000000 P 36f1f9a0d13f487ebeabdbbf1a07f259 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36f1f9a0d13f487ebeabdbbf1a07f259" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 46835 } }
I20260809 02:18:32.048810 4216 sys_catalog.cc:564] T 00000000000000000000000000000000 P 36f1f9a0d13f487ebeabdbbf1a07f259 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:18:32.049896 4225 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36f1f9a0d13f487ebeabdbbf1a07f259 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "36f1f9a0d13f487ebeabdbbf1a07f259" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36f1f9a0d13f487ebeabdbbf1a07f259" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 46835 } } }
I20260809 02:18:32.050024 4225 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36f1f9a0d13f487ebeabdbbf1a07f259 [sys.catalog]: This master's current role is: LEADER
I20260809 02:18:32.050347 4227 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36f1f9a0d13f487ebeabdbbf1a07f259 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 36f1f9a0d13f487ebeabdbbf1a07f259. Latest consensus state: current_term: 1 leader_uuid: "36f1f9a0d13f487ebeabdbbf1a07f259" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36f1f9a0d13f487ebeabdbbf1a07f259" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 46835 } } }
I20260809 02:18:32.050434 4238 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:18:32.050514 4227 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36f1f9a0d13f487ebeabdbbf1a07f259 [sys.catalog]: This master's current role is: LEADER
I20260809 02:18:32.053282 4238 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:18:32.060724 4238 catalog_manager.cc:1347] Generated new cluster ID: 49619f9c30c844d0a4fd80d5dc9257e9
I20260809 02:18:32.060791 4238 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:18:32.082016 4238 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:18:32.082940 4238 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:18:32.095610 4238 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 36f1f9a0d13f487ebeabdbbf1a07f259: Generated new TSK 0
I20260809 02:18:32.096280 4238 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:18:32.101305 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 4109
2026-08-09T02:18:32Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1626 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:18:32 dist-test-slave-s11t krb5kdc[4258](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:18:32 dist-test-slave-s11t krb5kdc[4258](info): set up 2 sockets
Aug 09 02:18:32 dist-test-slave-s11t krb5kdc[4258](info): commencing operation
krb5kdc: starting...
W20260809 02:18:33.349386 15662 mini_kdc.cc:121] Time spent starting KDC: real 1.221s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Aug 09 02:18:33 dist-test-slave-s11t krb5kdc[4258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241913, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:18:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:18:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.75.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab.
Entry for principal kudu/127.15.75.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab.
I20260809 02:18:33.473552 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:35621
--webserver_interface=127.15.75.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab
--principal=kudu/127.15.75.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:42113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.75.190:35621
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:18:33.582612 4276 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:18:33.582871 4276 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:18:33.582908 4276 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:18:33.586723 4276 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:18:33.586768 4276 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:18:33.586787 4276 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:18:33.586803 4276 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:18:33.586818 4276 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:18:33.591745 4276 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:42113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.75.190:35621
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab
--principal=kudu/127.15.75.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:35621
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.75.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4276
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:18:33.592634 4276 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:18:33.593626 4276 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:18:33.602103 4282 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:33.602100 4285 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:33.602077 4283 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:33.602433 4276 server_base.cc:1034] running on GCE node
I20260809 02:18:33.602963 4276 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:18:33.603960 4276 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:18:33.605149 4276 hybrid_clock.cc:648] HybridClock initialized: now 1786241913605130 us; error 31 us; skew 500 ppm
Aug 09 02:18:33 dist-test-slave-s11t krb5kdc[4258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241913, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:18:33.607858 4276 init.cc:377] Logged in from keytab as kudu/127.15.75.190@KRBTEST.COM (short username kudu)
I20260809 02:18:33.608745 4276 webserver.cc:460] Webserver started at http://127.15.75.190:45965/ using document root <none> and password file <none>
I20260809 02:18:33.609244 4276 fs_manager.cc:359] Metadata directory not provided
I20260809 02:18:33.609300 4276 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:18:33.609459 4276 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:18:33.611325 4276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/master-0/data/instance:
uuid: "84a663e56c0248d1afacc5b1b6bf0298"
format_stamp: "Formatted at 2026-08-09 02:18:33 on dist-test-slave-s11t"
server_key: "04e8b42f6479cf489e38a750b41650d7"
server_key_iv: "8a24440a2521a44c67a8ed3ab63a5a71"
server_key_version: "encryptionkey@0"
I20260809 02:18:33.611809 4276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/master-0/wal/instance:
uuid: "84a663e56c0248d1afacc5b1b6bf0298"
format_stamp: "Formatted at 2026-08-09 02:18:33 on dist-test-slave-s11t"
server_key: "04e8b42f6479cf489e38a750b41650d7"
server_key_iv: "8a24440a2521a44c67a8ed3ab63a5a71"
server_key_version: "encryptionkey@0"
I20260809 02:18:33.615279 4276 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260809 02:18:33.617518 4300 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:33.618713 4276 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 02:18:33.618831 4276 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/master-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/master-0/wal
uuid: "84a663e56c0248d1afacc5b1b6bf0298"
format_stamp: "Formatted at 2026-08-09 02:18:33 on dist-test-slave-s11t"
server_key: "04e8b42f6479cf489e38a750b41650d7"
server_key_iv: "8a24440a2521a44c67a8ed3ab63a5a71"
server_key_version: "encryptionkey@0"
I20260809 02:18:33.618938 4276 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:18:33.643831 4276 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:18:33.647395 4276 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:18:33.647590 4276 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:18:33.654541 4276 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.190:35621
I20260809 02:18:33.654841 4379 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.190:35621 every 8 connection(s)
I20260809 02:18:33.655483 4276 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
I20260809 02:18:33.658228 4382 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:18:33.659261 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 4276
I20260809 02:18:33.659389 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/master-0/wal/instance
I20260809 02:18:33.659686 15662 external_mini_cluster.cc:1428] Setting key 2ec29e054e53e562b4128d7a9e3c7afd
I20260809 02:18:33.664852 4382 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84a663e56c0248d1afacc5b1b6bf0298: Bootstrap starting.
I20260809 02:18:33.667356 4382 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 84a663e56c0248d1afacc5b1b6bf0298: Neither blocks nor log segments found. Creating new log.
I20260809 02:18:33.668066 4382 log.cc:824] T 00000000000000000000000000000000 P 84a663e56c0248d1afacc5b1b6bf0298: Log is configured to *not* fsync() on all Append() calls
Aug 09 02:18:33 dist-test-slave-s11t krb5kdc[4258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241913, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:18:33.670301 4382 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84a663e56c0248d1afacc5b1b6bf0298: No bootstrap required, opened a new log
I20260809 02:18:33.674381 4386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:33.661023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50338 (local address 127.15.75.190:35621)
0809 02:18:33.661426 (+ 403us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:33.661446 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:33.661494 (+ 48us) server_negotiation.cc:407] Connection header received
0809 02:18:33.662318 (+ 824us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:33.662342 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:33.662760 (+ 418us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:33.663119 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:33.663751 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:33.665182 (+ 1431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:33.666015 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:33.666666 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:33.666842 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:33.669306 (+ 2464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:33.669312 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:33.669327 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:18:33.669356 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:33.671258 (+ 1902us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:33.671669 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:33.671679 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:33.671685 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:33.671848 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:33.672103 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:33.672107 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:33.672108 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:33.672747 (+ 639us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:33.672947 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:33.673141 (+ 194us) server_negotiation.cc:299] Negotiation successful
0809 02:18:33.673436 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":107,"threads_started":1}
I20260809 02:18:33.674813 4382 raft_consensus.cc:348] T 00000000000000000000000000000000 P 84a663e56c0248d1afacc5b1b6bf0298 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84a663e56c0248d1afacc5b1b6bf0298" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 35621 } }
I20260809 02:18:33.675079 4382 raft_consensus.cc:374] T 00000000000000000000000000000000 P 84a663e56c0248d1afacc5b1b6bf0298 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:18:33.675176 4382 raft_consensus.cc:729] T 00000000000000000000000000000000 P 84a663e56c0248d1afacc5b1b6bf0298 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84a663e56c0248d1afacc5b1b6bf0298, State: Initialized, Role: FOLLOWER
I20260809 02:18:33.675622 4382 consensus_queue.cc:260] T 00000000000000000000000000000000 P 84a663e56c0248d1afacc5b1b6bf0298 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84a663e56c0248d1afacc5b1b6bf0298" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 35621 } }
I20260809 02:18:33.675750 4382 raft_consensus.cc:388] T 00000000000000000000000000000000 P 84a663e56c0248d1afacc5b1b6bf0298 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:18:33.675835 4382 raft_consensus.cc:482] T 00000000000000000000000000000000 P 84a663e56c0248d1afacc5b1b6bf0298 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:18:33.675959 4382 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 84a663e56c0248d1afacc5b1b6bf0298 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:18:33.676998 4382 raft_consensus.cc:504] T 00000000000000000000000000000000 P 84a663e56c0248d1afacc5b1b6bf0298 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84a663e56c0248d1afacc5b1b6bf0298" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 35621 } }
I20260809 02:18:33.677337 4382 leader_election.cc:302] T 00000000000000000000000000000000 P 84a663e56c0248d1afacc5b1b6bf0298 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 84a663e56c0248d1afacc5b1b6bf0298; no voters:
I20260809 02:18:33.677675 4382 leader_election.cc:288] T 00000000000000000000000000000000 P 84a663e56c0248d1afacc5b1b6bf0298 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:18:33.677762 4388 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 84a663e56c0248d1afacc5b1b6bf0298 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:18:33.678022 4388 raft_consensus.cc:686] T 00000000000000000000000000000000 P 84a663e56c0248d1afacc5b1b6bf0298 [term 1 LEADER]: Becoming Leader. State: Replica: 84a663e56c0248d1afacc5b1b6bf0298, State: Running, Role: LEADER
I20260809 02:18:33.678357 4388 consensus_queue.cc:237] T 00000000000000000000000000000000 P 84a663e56c0248d1afacc5b1b6bf0298 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84a663e56c0248d1afacc5b1b6bf0298" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 35621 } }
I20260809 02:18:33.678874 4382 sys_catalog.cc:564] T 00000000000000000000000000000000 P 84a663e56c0248d1afacc5b1b6bf0298 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:18:33.680070 4390 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84a663e56c0248d1afacc5b1b6bf0298 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "84a663e56c0248d1afacc5b1b6bf0298" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84a663e56c0248d1afacc5b1b6bf0298" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 35621 } } }
I20260809 02:18:33.680070 4391 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84a663e56c0248d1afacc5b1b6bf0298 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 84a663e56c0248d1afacc5b1b6bf0298. Latest consensus state: current_term: 1 leader_uuid: "84a663e56c0248d1afacc5b1b6bf0298" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84a663e56c0248d1afacc5b1b6bf0298" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 35621 } } }
I20260809 02:18:33.680228 4391 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84a663e56c0248d1afacc5b1b6bf0298 [sys.catalog]: This master's current role is: LEADER
I20260809 02:18:33.680228 4390 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84a663e56c0248d1afacc5b1b6bf0298 [sys.catalog]: This master's current role is: LEADER
I20260809 02:18:33.681129 4403 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:18:33.683627 4403 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:18:33.689069 4403 catalog_manager.cc:1347] Generated new cluster ID: 68aea2d3fcfe4278a84f4fdaa2e4afae
I20260809 02:18:33.689139 4403 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:18:33.702868 4403 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:18:33.703994 4403 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:18:33.713972 4403 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 84a663e56c0248d1afacc5b1b6bf0298: Generated new TSK 0
I20260809 02:18:33.714661 4403 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.75.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab.
Entry for principal kudu/127.15.75.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab.
I20260809 02:18:33.747009 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.129:0
--local_ip_for_outbound_sockets=127.15.75.129
--webserver_interface=127.15.75.129
--webserver_port=0
--tserver_master_addrs=127.15.75.190:35621
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab
--principal=kudu/127.15.75.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:42113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:18:33.856844 4424 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:18:33.857084 4424 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:18:33.857156 4424 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:18:33.861110 4424 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:18:33.861161 4424 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:18:33.861217 4424 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.129
I20260809 02:18:33.866238 4424 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:42113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab
--principal=kudu/127.15.75.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.75.129
--webserver_port=0
--tserver_master_addrs=127.15.75.190:35621
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4424
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.129
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:18:33.867229 4424 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:18:33.868229 4424 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:18:33.877697 4437 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:33.877693 4433 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:33.877751 4435 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:33.878152 4424 server_base.cc:1034] running on GCE node
I20260809 02:18:33.878473 4424 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:18:33.879058 4424 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:18:33.880249 4424 hybrid_clock.cc:648] HybridClock initialized: now 1786241913880227 us; error 34 us; skew 500 ppm
Aug 09 02:18:33 dist-test-slave-s11t krb5kdc[4258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241913, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:18:33.883114 4424 init.cc:377] Logged in from keytab as kudu/127.15.75.129@KRBTEST.COM (short username kudu)
I20260809 02:18:33.883930 4424 webserver.cc:460] Webserver started at http://127.15.75.129:40793/ using document root <none> and password file <none>
I20260809 02:18:33.884415 4424 fs_manager.cc:359] Metadata directory not provided
I20260809 02:18:33.884473 4424 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:18:33.884630 4424 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:18:33.886590 4424 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-0/data/instance:
uuid: "352b7fb566bf458585a84dd49f8c07dc"
format_stamp: "Formatted at 2026-08-09 02:18:33 on dist-test-slave-s11t"
server_key: "c40cb3aae85a366be328e883500d52f8"
server_key_iv: "6876e9e96b4ce7b076beac3486380a8b"
server_key_version: "encryptionkey@0"
I20260809 02:18:33.887092 4424 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-0/wal/instance:
uuid: "352b7fb566bf458585a84dd49f8c07dc"
format_stamp: "Formatted at 2026-08-09 02:18:33 on dist-test-slave-s11t"
server_key: "c40cb3aae85a366be328e883500d52f8"
server_key_iv: "6876e9e96b4ce7b076beac3486380a8b"
server_key_version: "encryptionkey@0"
I20260809 02:18:33.890537 4424 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 02:18:33.892851 4445 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:33.893981 4424 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 02:18:33.894074 4424 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-0/wal
uuid: "352b7fb566bf458585a84dd49f8c07dc"
format_stamp: "Formatted at 2026-08-09 02:18:33 on dist-test-slave-s11t"
server_key: "c40cb3aae85a366be328e883500d52f8"
server_key_iv: "6876e9e96b4ce7b076beac3486380a8b"
server_key_version: "encryptionkey@0"
I20260809 02:18:33.894150 4424 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:18:33.906347 4424 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:18:33.909324 4424 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:18:33.909466 4424 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:18:33.909934 4424 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:18:33.910665 4424 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:18:33.910710 4424 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:33.910746 4424 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:18:33.910761 4424 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:33.919621 4424 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.129:42815
I20260809 02:18:33.919651 4625 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.129:42815 every 8 connection(s)
I20260809 02:18:33.920420 4424 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
I20260809 02:18:33.923041 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 4424
I20260809 02:18:33.923171 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-0/wal/instance
I20260809 02:18:33.923489 15662 external_mini_cluster.cc:1428] Setting key ee269980c2701c41c902c2a97a2778d2
Aug 09 02:18:33 dist-test-slave-s11t krb5kdc[4258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241913, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.129@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.75.130@KRBTEST.COM; defaulting to no policy
I20260809 02:18:33.933962 4386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:33.922100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.129:35249 (local address 127.15.75.190:35621)
0809 02:18:33.922238 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:33.922243 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:33.923113 (+ 870us) server_negotiation.cc:407] Connection header received
0809 02:18:33.924590 (+ 1477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:33.924593 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:33.924646 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:33.924762 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:33.925831 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:33.926369 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:33.927477 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:33.927718 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:33.927828 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:33.930466 (+ 2638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:33.930469 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:33.930471 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:18:33.930494 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:33.931905 (+ 1411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:33.932407 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:33.932413 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:33.932418 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:33.932500 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:33.932905 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:33.932909 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:33.932910 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:33.933087 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:33.933197 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:33.933664 (+ 467us) server_negotiation.cc:299] Negotiation successful
0809 02:18:33.933782 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 02:18:33.934635 4629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:33.922459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:35621 (local address 127.15.75.129:35249)
0809 02:18:33.922965 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:33.923018 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:33.924389 (+ 1371us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:33.924918 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:33.924925 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:33.925248 (+ 323us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:18:33.925681 (+ 433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:33.925693 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:33.926501 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:33.926509 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:33.927354 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:33.927363 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:33.927866 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:33.927870 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:33.928114 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:33.928670 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:18:33.928721 (+ 51us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:18:33.930292 (+ 1571us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:18:33.932029 (+ 1737us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:18:33.932035 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:18:33.932047 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:18:33.932302 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:18:33.932597 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:18:33.932600 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:18:33.932602 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:18:33.932802 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:18:33.933197 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:18:33.933204 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:18:33.933438 (+ 234us) client_negotiation.cc:769] Sending connection context
0809 02:18:33.933618 (+ 180us) client_negotiation.cc:240] Negotiation successful
0809 02:18:33.933905 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":117,"threads_started":1}
I20260809 02:18:33.935843 4626 heartbeater.cc:347] Connected to a master server at 127.15.75.190:35621
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.130@KRBTEST.COM" created.
I20260809 02:18:33.936121 4626 heartbeater.cc:464] Registering TS with master...
I20260809 02:18:33.936739 4626 heartbeater.cc:511] Master 127.15.75.190:35621 requested a full tablet report, sending...
I20260809 02:18:33.938323 4319 ts_manager.cc:194] Registered new tserver with Master: 352b7fb566bf458585a84dd49f8c07dc (127.15.75.129:42815)
I20260809 02:18:33.939436 4319 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.75.129@KRBTEST.COM'} at 127.15.75.129:35249
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab.
Entry for principal kudu/127.15.75.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab.
I20260809 02:18:33.951593 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.130:0
--local_ip_for_outbound_sockets=127.15.75.130
--webserver_interface=127.15.75.130
--webserver_port=0
--tserver_master_addrs=127.15.75.190:35621
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab
--principal=kudu/127.15.75.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:42113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:18:34.061534 4633 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:18:34.061808 4633 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:18:34.061887 4633 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:18:34.065871 4633 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:18:34.065948 4633 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:18:34.066035 4633 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.130
I20260809 02:18:34.070963 4633 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:42113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab
--principal=kudu/127.15.75.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.75.130
--webserver_port=0
--tserver_master_addrs=127.15.75.190:35621
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4633
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.130
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:18:34.071902 4633 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:18:34.072909 4633 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:18:34.082885 4638 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:34.082861 4641 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:34.082880 4639 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:34.083282 4633 server_base.cc:1034] running on GCE node
I20260809 02:18:34.083597 4633 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:18:34.084208 4633 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:18:34.085372 4633 hybrid_clock.cc:648] HybridClock initialized: now 1786241914085346 us; error 46 us; skew 500 ppm
Aug 09 02:18:34 dist-test-slave-s11t krb5kdc[4258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241914, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:18:34.087955 4633 init.cc:377] Logged in from keytab as kudu/127.15.75.130@KRBTEST.COM (short username kudu)
I20260809 02:18:34.088881 4633 webserver.cc:460] Webserver started at http://127.15.75.130:42913/ using document root <none> and password file <none>
I20260809 02:18:34.089407 4633 fs_manager.cc:359] Metadata directory not provided
I20260809 02:18:34.089468 4633 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:18:34.089711 4633 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:18:34.091609 4633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-1/data/instance:
uuid: "d295396cdcae4b57acab477dfe1bd61d"
format_stamp: "Formatted at 2026-08-09 02:18:34 on dist-test-slave-s11t"
server_key: "f2f70264978910559b7a16b1d94caa58"
server_key_iv: "35a31c73d12962326cff4ca00a57858e"
server_key_version: "encryptionkey@0"
I20260809 02:18:34.092309 4633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-1/wal/instance:
uuid: "d295396cdcae4b57acab477dfe1bd61d"
format_stamp: "Formatted at 2026-08-09 02:18:34 on dist-test-slave-s11t"
server_key: "f2f70264978910559b7a16b1d94caa58"
server_key_iv: "35a31c73d12962326cff4ca00a57858e"
server_key_version: "encryptionkey@0"
I20260809 02:18:34.095897 4633 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260809 02:18:34.098294 4653 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:34.099469 4633 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 02:18:34.099607 4633 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-1/wal
uuid: "d295396cdcae4b57acab477dfe1bd61d"
format_stamp: "Formatted at 2026-08-09 02:18:34 on dist-test-slave-s11t"
server_key: "f2f70264978910559b7a16b1d94caa58"
server_key_iv: "35a31c73d12962326cff4ca00a57858e"
server_key_version: "encryptionkey@0"
I20260809 02:18:34.099753 4633 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:18:34.110754 4633 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:18:34.113484 4633 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:18:34.113675 4633 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:18:34.114214 4633 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:18:34.114982 4633 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:18:34.115053 4633 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:34.115124 4633 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:18:34.115168 4633 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:34.124307 4633 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.130:46695
I20260809 02:18:34.124342 4810 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.130:46695 every 8 connection(s)
I20260809 02:18:34.125188 4633 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
I20260809 02:18:34.128072 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 4633
I20260809 02:18:34.128201 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-1/wal/instance
I20260809 02:18:34.128580 15662 external_mini_cluster.cc:1428] Setting key d8dd284ebda33a7fb1503c9bf3668072
Aug 09 02:18:34 dist-test-slave-s11t krb5kdc[4258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241914, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.130@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.75.131@KRBTEST.COM; defaulting to no policy
I20260809 02:18:34.140324 4386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:34.126848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.130:58811 (local address 127.15.75.190:35621)
0809 02:18:34.126993 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:34.126997 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:34.127771 (+ 774us) server_negotiation.cc:407] Connection header received
0809 02:18:34.129654 (+ 1883us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:34.129657 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:34.129712 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:34.129813 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:34.131276 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:34.131787 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:34.133229 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:34.133608 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:34.133725 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:34.136408 (+ 2683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:34.136416 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:34.136419 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:18:34.136441 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:34.138218 (+ 1777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:34.138713 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:34.138719 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:34.138720 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:34.138816 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:34.139154 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:34.139159 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:34.139162 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:34.139409 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:34.139530 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:34.139996 (+ 466us) server_negotiation.cc:299] Negotiation successful
0809 02:18:34.140118 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 02:18:34.141033 4817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:34.127214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:35621 (local address 127.15.75.130:58811)
0809 02:18:34.127632 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:34.127671 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:34.129451 (+ 1780us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:34.129983 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:34.129990 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:34.130467 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:18:34.131120 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:34.131143 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:34.131920 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:34.131927 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:34.133110 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:34.133120 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:34.133759 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:34.133762 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:34.133907 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:34.134480 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:18:34.134496 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:18:34.136242 (+ 1746us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:18:34.138344 (+ 2102us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:18:34.138349 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:18:34.138360 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:18:34.138602 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:18:34.138918 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:18:34.138921 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:18:34.138922 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:18:34.139049 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:18:34.139526 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:18:34.139531 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:18:34.139749 (+ 218us) client_negotiation.cc:769] Sending connection context
0809 02:18:34.139901 (+ 152us) client_negotiation.cc:240] Negotiation successful
0809 02:18:34.140127 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":112,"threads_started":1}
I20260809 02:18:34.142117 4815 heartbeater.cc:347] Connected to a master server at 127.15.75.190:35621
I20260809 02:18:34.142431 4815 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.131@KRBTEST.COM" created.
I20260809 02:18:34.143054 4815 heartbeater.cc:511] Master 127.15.75.190:35621 requested a full tablet report, sending...
I20260809 02:18:34.144137 4319 ts_manager.cc:194] Registered new tserver with Master: d295396cdcae4b57acab477dfe1bd61d (127.15.75.130:46695)
I20260809 02:18:34.145002 4319 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.75.130@KRBTEST.COM'} at 127.15.75.130:58811
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab.
Entry for principal kudu/127.15.75.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab.
I20260809 02:18:34.157274 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.131:0
--local_ip_for_outbound_sockets=127.15.75.131
--webserver_interface=127.15.75.131
--webserver_port=0
--tserver_master_addrs=127.15.75.190:35621
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab
--principal=kudu/127.15.75.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:42113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:18:34.269207 4820 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:18:34.269474 4820 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:18:34.269550 4820 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:18:34.273531 4820 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:18:34.273615 4820 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:18:34.273712 4820 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.131
I20260809 02:18:34.278834 4820 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:42113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab
--principal=kudu/127.15.75.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.75.131
--webserver_port=0
--tserver_master_addrs=127.15.75.190:35621
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4820
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.131
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:18:34.279794 4820 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:18:34.280905 4820 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:18:34.290323 4828 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:34.290303 4831 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:34.290298 4827 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:34.290535 4820 server_base.cc:1034] running on GCE node
I20260809 02:18:34.290952 4820 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:18:34.291555 4820 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:18:34.292783 4820 hybrid_clock.cc:648] HybridClock initialized: now 1786241914292762 us; error 30 us; skew 500 ppm
Aug 09 02:18:34 dist-test-slave-s11t krb5kdc[4258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241914, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:18:34.295893 4820 init.cc:377] Logged in from keytab as kudu/127.15.75.131@KRBTEST.COM (short username kudu)
I20260809 02:18:34.296766 4820 webserver.cc:460] Webserver started at http://127.15.75.131:34579/ using document root <none> and password file <none>
I20260809 02:18:34.297399 4820 fs_manager.cc:359] Metadata directory not provided
I20260809 02:18:34.297465 4820 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:18:34.297700 4820 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:18:34.299708 4820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-2/data/instance:
uuid: "36d5cac71f2a4ef9bbfee682d6c3f712"
format_stamp: "Formatted at 2026-08-09 02:18:34 on dist-test-slave-s11t"
server_key: "417270fd0d930863d79dfbe3d92c882d"
server_key_iv: "88efa4f9f7d188aef74706e9b6e19694"
server_key_version: "encryptionkey@0"
I20260809 02:18:34.300263 4820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-2/wal/instance:
uuid: "36d5cac71f2a4ef9bbfee682d6c3f712"
format_stamp: "Formatted at 2026-08-09 02:18:34 on dist-test-slave-s11t"
server_key: "417270fd0d930863d79dfbe3d92c882d"
server_key_iv: "88efa4f9f7d188aef74706e9b6e19694"
server_key_version: "encryptionkey@0"
I20260809 02:18:34.303992 4820 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260809 02:18:34.306352 4841 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:34.307534 4820 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 02:18:34.307663 4820 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-2/wal
uuid: "36d5cac71f2a4ef9bbfee682d6c3f712"
format_stamp: "Formatted at 2026-08-09 02:18:34 on dist-test-slave-s11t"
server_key: "417270fd0d930863d79dfbe3d92c882d"
server_key_iv: "88efa4f9f7d188aef74706e9b6e19694"
server_key_version: "encryptionkey@0"
I20260809 02:18:34.307768 4820 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:18:34.348282 4820 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:18:34.351181 4820 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:18:34.351388 4820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:18:34.351972 4820 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:18:34.352854 4820 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:18:34.352913 4820 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:34.352995 4820 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:18:34.353051 4820 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:34.361827 4820 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.131:46171
I20260809 02:18:34.361919 5030 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.131:46171 every 8 connection(s)
I20260809 02:18:34.362742 4820 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
I20260809 02:18:34.363145 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 4820
I20260809 02:18:34.363266 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-2/wal/instance
I20260809 02:18:34.363551 15662 external_mini_cluster.cc:1428] Setting key 6b585ad727b92249fdb7d1c9f306a207
Aug 09 02:18:34 dist-test-slave-s11t krb5kdc[4258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241914, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.131@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:18:34.376005 4386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:34.364635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.131:45377 (local address 127.15.75.190:35621)
0809 02:18:34.364774 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:34.364780 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:34.365893 (+ 1113us) server_negotiation.cc:407] Connection header received
0809 02:18:34.367245 (+ 1352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:34.367248 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:34.367296 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:34.367370 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:34.368418 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:34.368973 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:34.370066 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:34.370286 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:34.370370 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:34.372739 (+ 2369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:34.372743 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:34.372746 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:18:34.372768 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:34.374090 (+ 1322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:34.374584 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:34.374590 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:34.374593 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:34.374735 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:34.375053 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:34.375056 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:34.375057 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:34.375226 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:34.375304 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:34.375742 (+ 438us) server_negotiation.cc:299] Negotiation successful
0809 02:18:34.375834 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 02:18:34.376801 5033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:34.365121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:35621 (local address 127.15.75.131:45377)
0809 02:18:34.365613 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:34.365784 (+ 171us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:34.367044 (+ 1260us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:34.367535 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:34.367541 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:34.367861 (+ 320us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:18:34.368296 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:34.368308 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:34.369079 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:34.369082 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:34.369943 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:34.369952 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:34.370424 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:34.370427 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:34.370561 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:34.371124 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:18:34.371140 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:18:34.372531 (+ 1391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:18:34.374201 (+ 1670us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:18:34.374205 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:18:34.374215 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:18:34.374458 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:18:34.374831 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:18:34.374833 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:18:34.374834 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:18:34.374969 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:18:34.375322 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:18:34.375327 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:18:34.375527 (+ 200us) client_negotiation.cc:769] Sending connection context
0809 02:18:34.375672 (+ 145us) client_negotiation.cc:240] Negotiation successful
0809 02:18:34.375889 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":140,"threads_started":1}
I20260809 02:18:34.377918 5031 heartbeater.cc:347] Connected to a master server at 127.15.75.190:35621
I20260809 02:18:34.378157 5031 heartbeater.cc:464] Registering TS with master...
I20260809 02:18:34.378623 5031 heartbeater.cc:511] Master 127.15.75.190:35621 requested a full tablet report, sending...
I20260809 02:18:34.379647 4318 ts_manager.cc:194] Registered new tserver with Master: 36d5cac71f2a4ef9bbfee682d6c3f712 (127.15.75.131:46171)
I20260809 02:18:34.380167 4318 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.75.131@KRBTEST.COM'} at 127.15.75.131:45377
I20260809 02:18:34.389549 15662 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 02:18:34.398607 4386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:34.391516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42304 (local address 127.15.75.190:35621)
0809 02:18:34.391670 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:34.391674 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:34.391778 (+ 104us) server_negotiation.cc:407] Connection header received
0809 02:18:34.391878 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:34.391880 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:34.391922 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:34.391987 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:34.392829 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:34.393309 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:34.394087 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:34.394281 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:34.394354 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:34.395521 (+ 1167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:34.395523 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:34.395525 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:18:34.395545 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:34.396889 (+ 1344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:34.397382 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:34.397387 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:34.397389 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:34.397474 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:34.397750 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:34.397754 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:34.397755 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:34.397966 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:34.398109 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:34.398350 (+ 241us) server_negotiation.cc:299] Negotiation successful
0809 02:18:34.398461 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 02:18:34.401520 4318 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42304:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:18:34.404471 4318 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:18:34.418483 5055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:34.412927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.129:42815 (local address 127.0.0.1:40858)
0809 02:18:34.413451 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:34.413466 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:34.413557 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:34.414289 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:34.414293 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:34.414309 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:18:34.414430 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:34.414437 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:34.415821 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:34.415825 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:34.416825 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:34.416838 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:34.417953 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:34.417957 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:34.418074 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:34.418116 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 02:18:34.418215 (+ 99us) client_negotiation.cc:240] Negotiation successful
0809 02:18:34.418302 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"mutex_wait_us":19,"thread_start_us":96,"threads_started":1}
I20260809 02:18:34.418505 5059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:34.413364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40858 (local address 127.15.75.129:42815)
0809 02:18:34.413783 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:34.413827 (+ 44us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:34.413851 (+ 24us) server_negotiation.cc:407] Connection header received
0809 02:18:34.413929 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:34.413934 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:34.414112 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:34.414586 (+ 474us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:18:34.414739 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:34.415680 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:34.416976 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:34.417800 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:34.418007 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:34.418145 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:34.418245 (+ 100us) server_negotiation.cc:299] Negotiation successful
0809 02:18:34.418355 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":132,"threads_started":1}
I20260809 02:18:34.418809 5056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:34.413219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.131:46171 (local address 127.0.0.1:60702)
0809 02:18:34.414284 (+ 1065us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:34.414298 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:34.414385 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:34.414872 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:34.414875 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:34.414894 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:18:34.415000 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:34.415006 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:34.416225 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:34.416228 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:34.417337 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:34.417347 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:34.418387 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:34.418391 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:34.418482 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:34.418517 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 02:18:34.418596 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 02:18:34.418673 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1001,"thread_start_us":111,"threads_started":1}
I20260809 02:18:34.419225 5063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:34.413638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60702 (local address 127.15.75.131:46171)
0809 02:18:34.413983 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:34.413993 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:34.414490 (+ 497us) server_negotiation.cc:407] Connection header received
0809 02:18:34.414565 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:34.414570 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:34.414729 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:34.414865 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:18:34.415121 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:34.416082 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:34.417463 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:34.418257 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:34.418362 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:34.418497 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:34.419000 (+ 503us) server_negotiation.cc:299] Negotiation successful
0809 02:18:34.419088 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":253,"thread_start_us":137,"threads_started":1}
I20260809 02:18:34.419543 5053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:34.412608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35808 (local address 127.15.75.130:46695)
0809 02:18:34.413203 (+ 595us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:34.413216 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:34.413239 (+ 23us) server_negotiation.cc:407] Connection header received
0809 02:18:34.413324 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:34.413332 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:34.413545 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:34.413705 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:18:34.414232 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:34.415376 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:34.417017 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:34.417737 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:34.417871 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:34.418023 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:34.419247 (+ 1224us) server_negotiation.cc:299] Negotiation successful
0809 02:18:34.419361 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":492,"thread_start_us":156,"threads_started":1}
I20260809 02:18:34.419152 5052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:34.412475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.130:46695 (local address 127.0.0.1:35808)
0809 02:18:34.412895 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:34.412946 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:34.413077 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:34.413932 (+ 855us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:34.413938 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:34.413962 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:18:34.414096 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:34.414103 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:34.415515 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:34.415518 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:34.416874 (+ 1356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:34.416886 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:34.417872 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:34.417876 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:34.418848 (+ 972us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:34.418884 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 02:18:34.418963 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 02:18:34.419032 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"mutex_wait_us":34,"thread_start_us":91,"threads_started":1}
I20260809 02:18:34.421988 4907 tablet_service.cc:1467] Processing CreateTablet for tablet adc1ba9b16534bf7ad003c8946235627 (DEFAULT_TABLE table=test-table [id=d36b6bd551164a73ab85b5e95fe4557b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:18:34.421808 4509 tablet_service.cc:1467] Processing CreateTablet for tablet adc1ba9b16534bf7ad003c8946235627 (DEFAULT_TABLE table=test-table [id=d36b6bd551164a73ab85b5e95fe4557b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:18:34.423137 4907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet adc1ba9b16534bf7ad003c8946235627. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:18:34.423326 4509 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet adc1ba9b16534bf7ad003c8946235627. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:18:34.424391 4718 tablet_service.cc:1467] Processing CreateTablet for tablet adc1ba9b16534bf7ad003c8946235627 (DEFAULT_TABLE table=test-table [id=d36b6bd551164a73ab85b5e95fe4557b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:18:34.425374 4718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet adc1ba9b16534bf7ad003c8946235627. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:18:34.429620 5066 tablet_bootstrap.cc:492] T adc1ba9b16534bf7ad003c8946235627 P 352b7fb566bf458585a84dd49f8c07dc: Bootstrap starting.
I20260809 02:18:34.430600 5069 tablet_bootstrap.cc:492] T adc1ba9b16534bf7ad003c8946235627 P 36d5cac71f2a4ef9bbfee682d6c3f712: Bootstrap starting.
I20260809 02:18:34.431891 5070 tablet_bootstrap.cc:492] T adc1ba9b16534bf7ad003c8946235627 P d295396cdcae4b57acab477dfe1bd61d: Bootstrap starting.
I20260809 02:18:34.431972 5066 tablet_bootstrap.cc:654] T adc1ba9b16534bf7ad003c8946235627 P 352b7fb566bf458585a84dd49f8c07dc: Neither blocks nor log segments found. Creating new log.
I20260809 02:18:34.433079 5069 tablet_bootstrap.cc:654] T adc1ba9b16534bf7ad003c8946235627 P 36d5cac71f2a4ef9bbfee682d6c3f712: Neither blocks nor log segments found. Creating new log.
I20260809 02:18:34.433445 5066 log.cc:824] T adc1ba9b16534bf7ad003c8946235627 P 352b7fb566bf458585a84dd49f8c07dc: Log is configured to *not* fsync() on all Append() calls
I20260809 02:18:34.434284 5070 tablet_bootstrap.cc:654] T adc1ba9b16534bf7ad003c8946235627 P d295396cdcae4b57acab477dfe1bd61d: Neither blocks nor log segments found. Creating new log.
I20260809 02:18:34.434327 5069 log.cc:824] T adc1ba9b16534bf7ad003c8946235627 P 36d5cac71f2a4ef9bbfee682d6c3f712: Log is configured to *not* fsync() on all Append() calls
I20260809 02:18:34.435206 5070 log.cc:824] T adc1ba9b16534bf7ad003c8946235627 P d295396cdcae4b57acab477dfe1bd61d: Log is configured to *not* fsync() on all Append() calls
I20260809 02:18:34.435925 5066 tablet_bootstrap.cc:492] T adc1ba9b16534bf7ad003c8946235627 P 352b7fb566bf458585a84dd49f8c07dc: No bootstrap required, opened a new log
I20260809 02:18:34.436141 5066 ts_tablet_manager.cc:1397] T adc1ba9b16534bf7ad003c8946235627 P 352b7fb566bf458585a84dd49f8c07dc: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260809 02:18:34.437155 5070 tablet_bootstrap.cc:492] T adc1ba9b16534bf7ad003c8946235627 P d295396cdcae4b57acab477dfe1bd61d: No bootstrap required, opened a new log
I20260809 02:18:34.437160 5069 tablet_bootstrap.cc:492] T adc1ba9b16534bf7ad003c8946235627 P 36d5cac71f2a4ef9bbfee682d6c3f712: No bootstrap required, opened a new log
I20260809 02:18:34.437350 5070 ts_tablet_manager.cc:1397] T adc1ba9b16534bf7ad003c8946235627 P d295396cdcae4b57acab477dfe1bd61d: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260809 02:18:34.437461 5069 ts_tablet_manager.cc:1397] T adc1ba9b16534bf7ad003c8946235627 P 36d5cac71f2a4ef9bbfee682d6c3f712: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260809 02:18:34.439129 5066 raft_consensus.cc:348] T adc1ba9b16534bf7ad003c8946235627 P 352b7fb566bf458585a84dd49f8c07dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d295396cdcae4b57acab477dfe1bd61d" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 46695 } } peers { permanent_uuid: "36d5cac71f2a4ef9bbfee682d6c3f712" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 46171 } } peers { permanent_uuid: "352b7fb566bf458585a84dd49f8c07dc" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 42815 } }
I20260809 02:18:34.439361 5066 raft_consensus.cc:374] T adc1ba9b16534bf7ad003c8946235627 P 352b7fb566bf458585a84dd49f8c07dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:18:34.439459 5066 raft_consensus.cc:729] T adc1ba9b16534bf7ad003c8946235627 P 352b7fb566bf458585a84dd49f8c07dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 352b7fb566bf458585a84dd49f8c07dc, State: Initialized, Role: FOLLOWER
I20260809 02:18:34.439921 5066 consensus_queue.cc:260] T adc1ba9b16534bf7ad003c8946235627 P 352b7fb566bf458585a84dd49f8c07dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d295396cdcae4b57acab477dfe1bd61d" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 46695 } } peers { permanent_uuid: "36d5cac71f2a4ef9bbfee682d6c3f712" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 46171 } } peers { permanent_uuid: "352b7fb566bf458585a84dd49f8c07dc" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 42815 } }
I20260809 02:18:34.440488 4626 heartbeater.cc:503] Master 127.15.75.190:35621 was elected leader, sending a full tablet report...
I20260809 02:18:34.440881 5066 ts_tablet_manager.cc:1428] T adc1ba9b16534bf7ad003c8946235627 P 352b7fb566bf458585a84dd49f8c07dc: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260809 02:18:34.441166 5070 raft_consensus.cc:348] T adc1ba9b16534bf7ad003c8946235627 P d295396cdcae4b57acab477dfe1bd61d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d295396cdcae4b57acab477dfe1bd61d" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 46695 } } peers { permanent_uuid: "36d5cac71f2a4ef9bbfee682d6c3f712" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 46171 } } peers { permanent_uuid: "352b7fb566bf458585a84dd49f8c07dc" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 42815 } }
I20260809 02:18:34.441166 5069 raft_consensus.cc:348] T adc1ba9b16534bf7ad003c8946235627 P 36d5cac71f2a4ef9bbfee682d6c3f712 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d295396cdcae4b57acab477dfe1bd61d" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 46695 } } peers { permanent_uuid: "36d5cac71f2a4ef9bbfee682d6c3f712" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 46171 } } peers { permanent_uuid: "352b7fb566bf458585a84dd49f8c07dc" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 42815 } }
I20260809 02:18:34.441424 5069 raft_consensus.cc:374] T adc1ba9b16534bf7ad003c8946235627 P 36d5cac71f2a4ef9bbfee682d6c3f712 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:18:34.441504 5069 raft_consensus.cc:729] T adc1ba9b16534bf7ad003c8946235627 P 36d5cac71f2a4ef9bbfee682d6c3f712 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36d5cac71f2a4ef9bbfee682d6c3f712, State: Initialized, Role: FOLLOWER
I20260809 02:18:34.441691 5070 raft_consensus.cc:374] T adc1ba9b16534bf7ad003c8946235627 P d295396cdcae4b57acab477dfe1bd61d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:18:34.441810 5070 raft_consensus.cc:729] T adc1ba9b16534bf7ad003c8946235627 P d295396cdcae4b57acab477dfe1bd61d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d295396cdcae4b57acab477dfe1bd61d, State: Initialized, Role: FOLLOWER
I20260809 02:18:34.441974 5069 consensus_queue.cc:260] T adc1ba9b16534bf7ad003c8946235627 P 36d5cac71f2a4ef9bbfee682d6c3f712 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d295396cdcae4b57acab477dfe1bd61d" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 46695 } } peers { permanent_uuid: "36d5cac71f2a4ef9bbfee682d6c3f712" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 46171 } } peers { permanent_uuid: "352b7fb566bf458585a84dd49f8c07dc" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 42815 } }
I20260809 02:18:34.442282 5070 consensus_queue.cc:260] T adc1ba9b16534bf7ad003c8946235627 P d295396cdcae4b57acab477dfe1bd61d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d295396cdcae4b57acab477dfe1bd61d" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 46695 } } peers { permanent_uuid: "36d5cac71f2a4ef9bbfee682d6c3f712" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 46171 } } peers { permanent_uuid: "352b7fb566bf458585a84dd49f8c07dc" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 42815 } }
I20260809 02:18:34.442667 5031 heartbeater.cc:503] Master 127.15.75.190:35621 was elected leader, sending a full tablet report...
I20260809 02:18:34.442970 5069 ts_tablet_manager.cc:1428] T adc1ba9b16534bf7ad003c8946235627 P 36d5cac71f2a4ef9bbfee682d6c3f712: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260809 02:18:34.443228 5070 ts_tablet_manager.cc:1428] T adc1ba9b16534bf7ad003c8946235627 P d295396cdcae4b57acab477dfe1bd61d: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 02:18:34.443552 4815 heartbeater.cc:503] Master 127.15.75.190:35621 was elected leader, sending a full tablet report...
I20260809 02:18:34.561522 5075 raft_consensus.cc:482] T adc1ba9b16534bf7ad003c8946235627 P 352b7fb566bf458585a84dd49f8c07dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:18:34.561762 5075 raft_consensus.cc:504] T adc1ba9b16534bf7ad003c8946235627 P 352b7fb566bf458585a84dd49f8c07dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d295396cdcae4b57acab477dfe1bd61d" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 46695 } } peers { permanent_uuid: "36d5cac71f2a4ef9bbfee682d6c3f712" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 46171 } } peers { permanent_uuid: "352b7fb566bf458585a84dd49f8c07dc" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 42815 } }
I20260809 02:18:34.562887 5075 leader_election.cc:288] T adc1ba9b16534bf7ad003c8946235627 P 352b7fb566bf458585a84dd49f8c07dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d295396cdcae4b57acab477dfe1bd61d (127.15.75.130:46695), 36d5cac71f2a4ef9bbfee682d6c3f712 (127.15.75.131:46171)
I20260809 02:18:34.568125 5063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:34.563791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.129:42283 (local address 127.15.75.131:46171)
0809 02:18:34.563964 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:34.563970 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:34.564103 (+ 133us) server_negotiation.cc:407] Connection header received
0809 02:18:34.564206 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:34.564208 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:34.564247 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:34.564338 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:18:34.564588 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:34.565321 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:34.566842 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:34.567446 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:34.567571 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:34.567622 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:34.567941 (+ 319us) server_negotiation.cc:299] Negotiation successful
0809 02:18:34.568007 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 02:18:34.568125 5053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:34.563276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.129:39165 (local address 127.15.75.130:46695)
0809 02:18:34.563416 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:34.563420 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:34.563622 (+ 202us) server_negotiation.cc:407] Connection header received
0809 02:18:34.563823 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:34.563827 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:34.563888 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:34.563981 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:18:34.564349 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:34.565209 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:34.566836 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:34.567452 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:34.567571 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:34.567622 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:34.567922 (+ 300us) server_negotiation.cc:299] Negotiation successful
0809 02:18:34.567997 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 02:18:34.568226 5081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:34.563638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.131:46171 (local address 127.15.75.129:42283)
0809 02:18:34.564002 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:34.564015 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:34.564126 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:34.564373 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:34.564375 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:34.564392 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:18:34.564481 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:34.564485 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:34.565431 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:34.565434 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:34.566708 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:34.566713 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:34.567589 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:34.567592 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:34.567808 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:34.567837 (+ 29us) client_negotiation.cc:769] Sending connection context
0809 02:18:34.567904 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 02:18:34.568026 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"mutex_wait_us":177,"spinlock_wait_cycles":3072,"thread_start_us":113,"threads_started":1}
I20260809 02:18:34.568228 5079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:34.563178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.130:46695 (local address 127.15.75.129:39165)
0809 02:18:34.563521 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:34.563539 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:34.563650 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:34.564008 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:34.564014 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:34.564039 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:18:34.564220 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:34.564227 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:34.565326 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:34.565329 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:34.566701 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:34.566713 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:34.567589 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:34.567592 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:34.567763 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:34.567792 (+ 29us) client_negotiation.cc:769] Sending connection context
0809 02:18:34.567877 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 02:18:34.568021 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"mutex_wait_us":107,"spinlock_wait_cycles":24320,"thread_start_us":163,"threads_started":1}
I20260809 02:18:34.568950 4945 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "adc1ba9b16534bf7ad003c8946235627" candidate_uuid: "352b7fb566bf458585a84dd49f8c07dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d5cac71f2a4ef9bbfee682d6c3f712" is_pre_election: true
I20260809 02:18:34.569028 4744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "adc1ba9b16534bf7ad003c8946235627" candidate_uuid: "352b7fb566bf458585a84dd49f8c07dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d295396cdcae4b57acab477dfe1bd61d" is_pre_election: true
I20260809 02:18:34.569280 4945 raft_consensus.cc:2454] T adc1ba9b16534bf7ad003c8946235627 P 36d5cac71f2a4ef9bbfee682d6c3f712 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 352b7fb566bf458585a84dd49f8c07dc in term 0.
I20260809 02:18:34.569327 4744 raft_consensus.cc:2454] T adc1ba9b16534bf7ad003c8946235627 P d295396cdcae4b57acab477dfe1bd61d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 352b7fb566bf458585a84dd49f8c07dc in term 0.
I20260809 02:18:34.569737 4448 leader_election.cc:302] T adc1ba9b16534bf7ad003c8946235627 P 352b7fb566bf458585a84dd49f8c07dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 352b7fb566bf458585a84dd49f8c07dc, 36d5cac71f2a4ef9bbfee682d6c3f712; no voters:
I20260809 02:18:34.570030 5075 raft_consensus.cc:2781] T adc1ba9b16534bf7ad003c8946235627 P 352b7fb566bf458585a84dd49f8c07dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:18:34.570122 5075 raft_consensus.cc:482] T adc1ba9b16534bf7ad003c8946235627 P 352b7fb566bf458585a84dd49f8c07dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:18:34.570169 5075 raft_consensus.cc:3037] T adc1ba9b16534bf7ad003c8946235627 P 352b7fb566bf458585a84dd49f8c07dc [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:18:34.571225 5075 raft_consensus.cc:504] T adc1ba9b16534bf7ad003c8946235627 P 352b7fb566bf458585a84dd49f8c07dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d295396cdcae4b57acab477dfe1bd61d" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 46695 } } peers { permanent_uuid: "36d5cac71f2a4ef9bbfee682d6c3f712" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 46171 } } peers { permanent_uuid: "352b7fb566bf458585a84dd49f8c07dc" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 42815 } }
I20260809 02:18:34.571563 5075 leader_election.cc:288] T adc1ba9b16534bf7ad003c8946235627 P 352b7fb566bf458585a84dd49f8c07dc [CANDIDATE]: Term 1 election: Requested vote from peers d295396cdcae4b57acab477dfe1bd61d (127.15.75.130:46695), 36d5cac71f2a4ef9bbfee682d6c3f712 (127.15.75.131:46171)
I20260809 02:18:34.571934 4744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "adc1ba9b16534bf7ad003c8946235627" candidate_uuid: "352b7fb566bf458585a84dd49f8c07dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d295396cdcae4b57acab477dfe1bd61d"
I20260809 02:18:34.571961 4945 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "adc1ba9b16534bf7ad003c8946235627" candidate_uuid: "352b7fb566bf458585a84dd49f8c07dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d5cac71f2a4ef9bbfee682d6c3f712"
I20260809 02:18:34.572037 4744 raft_consensus.cc:3037] T adc1ba9b16534bf7ad003c8946235627 P d295396cdcae4b57acab477dfe1bd61d [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:18:34.572088 4945 raft_consensus.cc:3037] T adc1ba9b16534bf7ad003c8946235627 P 36d5cac71f2a4ef9bbfee682d6c3f712 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:18:34.573186 4945 raft_consensus.cc:2454] T adc1ba9b16534bf7ad003c8946235627 P 36d5cac71f2a4ef9bbfee682d6c3f712 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 352b7fb566bf458585a84dd49f8c07dc in term 1.
I20260809 02:18:34.573189 4744 raft_consensus.cc:2454] T adc1ba9b16534bf7ad003c8946235627 P d295396cdcae4b57acab477dfe1bd61d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 352b7fb566bf458585a84dd49f8c07dc in term 1.
I20260809 02:18:34.573454 4448 leader_election.cc:302] T adc1ba9b16534bf7ad003c8946235627 P 352b7fb566bf458585a84dd49f8c07dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 352b7fb566bf458585a84dd49f8c07dc, 36d5cac71f2a4ef9bbfee682d6c3f712; no voters:
I20260809 02:18:34.573632 5075 raft_consensus.cc:2781] T adc1ba9b16534bf7ad003c8946235627 P 352b7fb566bf458585a84dd49f8c07dc [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:18:34.573848 5075 raft_consensus.cc:686] T adc1ba9b16534bf7ad003c8946235627 P 352b7fb566bf458585a84dd49f8c07dc [term 1 LEADER]: Becoming Leader. State: Replica: 352b7fb566bf458585a84dd49f8c07dc, State: Running, Role: LEADER
I20260809 02:18:34.574162 5075 consensus_queue.cc:237] T adc1ba9b16534bf7ad003c8946235627 P 352b7fb566bf458585a84dd49f8c07dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d295396cdcae4b57acab477dfe1bd61d" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 46695 } } peers { permanent_uuid: "36d5cac71f2a4ef9bbfee682d6c3f712" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 46171 } } peers { permanent_uuid: "352b7fb566bf458585a84dd49f8c07dc" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 42815 } }
I20260809 02:18:34.577266 4319 catalog_manager.cc:5495] T adc1ba9b16534bf7ad003c8946235627 P 352b7fb566bf458585a84dd49f8c07dc reported cstate change: term changed from 0 to 1, leader changed from <none> to 352b7fb566bf458585a84dd49f8c07dc (127.15.75.129). New cstate: current_term: 1 leader_uuid: "352b7fb566bf458585a84dd49f8c07dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d295396cdcae4b57acab477dfe1bd61d" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 46695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36d5cac71f2a4ef9bbfee682d6c3f712" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 46171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "352b7fb566bf458585a84dd49f8c07dc" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 42815 } health_report { overall_health: HEALTHY } } }
I20260809 02:18:34.605756 5059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:34.602104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40862 (local address 127.15.75.129:42815)
0809 02:18:34.602245 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:34.602249 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:34.602263 (+ 14us) server_negotiation.cc:407] Connection header received
0809 02:18:34.602422 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:34.602425 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:34.602480 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:34.602588 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:18:34.602884 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:34.603400 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:34.604307 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:34.604528 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:34.604637 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:34.604973 (+ 336us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:18:34.605361 (+ 388us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:18:34.605495 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:34.605561 (+ 66us) server_negotiation.cc:299] Negotiation successful
0809 02:18:34.605617 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 02:18:34.611281 4744 raft_consensus.cc:1261] T adc1ba9b16534bf7ad003c8946235627 P d295396cdcae4b57acab477dfe1bd61d [term 1 FOLLOWER]: Refusing update from remote peer 352b7fb566bf458585a84dd49f8c07dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:18:34.611307 4945 raft_consensus.cc:1261] T adc1ba9b16534bf7ad003c8946235627 P 36d5cac71f2a4ef9bbfee682d6c3f712 [term 1 FOLLOWER]: Refusing update from remote peer 352b7fb566bf458585a84dd49f8c07dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:18:34.612011 5075 consensus_queue.cc:1035] T adc1ba9b16534bf7ad003c8946235627 P 352b7fb566bf458585a84dd49f8c07dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "36d5cac71f2a4ef9bbfee682d6c3f712" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 46171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:18:34.612186 5082 consensus_queue.cc:1035] T adc1ba9b16534bf7ad003c8946235627 P 352b7fb566bf458585a84dd49f8c07dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "d295396cdcae4b57acab477dfe1bd61d" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 46695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 02:18:34.613984 5032 tablet.cc:2369] T adc1ba9b16534bf7ad003c8946235627 P 36d5cac71f2a4ef9bbfee682d6c3f712: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:18:34.619692 5088 mvcc.cc:205] Tried to move back new op lower bound from 7316446882240897024 to 7316446882098360320. Current Snapshot: MvccSnapshot[applied={T|T < 7316446882240897024}]
I20260809 02:18:34.628351 4319 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42304:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:18:34.628510 4319 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42304:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:18:34.630760 4319 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 84a663e56c0248d1afacc5b1b6bf0298: Sending DeleteTablet for 3 replicas of tablet adc1ba9b16534bf7ad003c8946235627
I20260809 02:18:34.631570 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 4424
I20260809 02:18:34.631531 4509 tablet_service.cc:1514] Processing DeleteTablet for tablet adc1ba9b16534bf7ad003c8946235627 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:18:34 UTC) from {username='kudu', principal='kudu/127.15.75.190@KRBTEST.COM'} at 127.0.0.1:40858
I20260809 02:18:34.631656 4907 tablet_service.cc:1514] Processing DeleteTablet for tablet adc1ba9b16534bf7ad003c8946235627 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:18:34 UTC) from {username='kudu', principal='kudu/127.15.75.190@KRBTEST.COM'} at 127.0.0.1:60702
I20260809 02:18:34.634310 4718 tablet_service.cc:1514] Processing DeleteTablet for tablet adc1ba9b16534bf7ad003c8946235627 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:18:34 UTC) from {username='kudu', principal='kudu/127.15.75.190@KRBTEST.COM'} at 127.0.0.1:35808
I20260809 02:18:34.635466 5102 tablet_replica.cc:330] T adc1ba9b16534bf7ad003c8946235627 P d295396cdcae4b57acab477dfe1bd61d: stopping tablet replica
I20260809 02:18:34.635793 5102 raft_consensus.cc:2229] T adc1ba9b16534bf7ad003c8946235627 P d295396cdcae4b57acab477dfe1bd61d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:18:34.636101 5102 raft_consensus.cc:2258] T adc1ba9b16534bf7ad003c8946235627 P d295396cdcae4b57acab477dfe1bd61d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:18:34.639070 5102 ts_tablet_manager.cc:1905] T adc1ba9b16534bf7ad003c8946235627 P d295396cdcae4b57acab477dfe1bd61d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:18:34.642416 5102 ts_tablet_manager.cc:1918] T adc1ba9b16534bf7ad003c8946235627 P d295396cdcae4b57acab477dfe1bd61d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:18:34.642535 5102 log.cc:1196] T adc1ba9b16534bf7ad003c8946235627 P d295396cdcae4b57acab477dfe1bd61d: Deleting WAL directory at /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-1/wal/wals/adc1ba9b16534bf7ad003c8946235627
I20260809 02:18:34.642890 5102 ts_tablet_manager.cc:1939] T adc1ba9b16534bf7ad003c8946235627 P d295396cdcae4b57acab477dfe1bd61d: Deleting consensus metadata
I20260809 02:18:34.643971 4302 catalog_manager.cc:4841] TS d295396cdcae4b57acab477dfe1bd61d (127.15.75.130:46695): tablet adc1ba9b16534bf7ad003c8946235627 (table test-table [id=d36b6bd551164a73ab85b5e95fe4557b]) successfully deleted
I20260809 02:18:34.645045 5101 tablet_replica.cc:330] T adc1ba9b16534bf7ad003c8946235627 P 36d5cac71f2a4ef9bbfee682d6c3f712: stopping tablet replica
I20260809 02:18:34.645388 5101 raft_consensus.cc:2229] T adc1ba9b16534bf7ad003c8946235627 P 36d5cac71f2a4ef9bbfee682d6c3f712 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:18:34.645714 5101 raft_consensus.cc:2258] T adc1ba9b16534bf7ad003c8946235627 P 36d5cac71f2a4ef9bbfee682d6c3f712 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:18:34.647141 5101 ts_tablet_manager.cc:1905] T adc1ba9b16534bf7ad003c8946235627 P 36d5cac71f2a4ef9bbfee682d6c3f712: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:18:34.650574 5101 ts_tablet_manager.cc:1918] T adc1ba9b16534bf7ad003c8946235627 P 36d5cac71f2a4ef9bbfee682d6c3f712: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:18:34.650688 5101 log.cc:1196] T adc1ba9b16534bf7ad003c8946235627 P 36d5cac71f2a4ef9bbfee682d6c3f712: Deleting WAL directory at /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-2/wal/wals/adc1ba9b16534bf7ad003c8946235627
I20260809 02:18:34.651053 5101 ts_tablet_manager.cc:1939] T adc1ba9b16534bf7ad003c8946235627 P 36d5cac71f2a4ef9bbfee682d6c3f712: Deleting consensus metadata
I20260809 02:18:34.652289 4302 catalog_manager.cc:4841] TS 36d5cac71f2a4ef9bbfee682d6c3f712 (127.15.75.131:46171): tablet adc1ba9b16534bf7ad003c8946235627 (table test-table [id=d36b6bd551164a73ab85b5e95fe4557b]) successfully deleted
I20260809 02:18:34.652719 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 4633
W20260809 02:18:34.652889 4304 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.75.129:42815 (error 108)
I20260809 02:18:34.661598 5052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:34.661337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.129:42815 (local address 127.0.0.1:40870)
0809 02:18:34.661463 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:34.661534 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.129:42815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 02:18:34.661827 4304 catalog_manager.cc:4584] TS 352b7fb566bf458585a84dd49f8c07dc (127.15.75.129:42815): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet adc1ba9b16534bf7ad003c8946235627: Network error: Client connection negotiation failed: client connection to 127.15.75.129:42815: connect: Connection refused (error 111)
I20260809 02:18:34.663909 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 4820
I20260809 02:18:34.675009 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 4276
I20260809 02:18:34.685458 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:35621
--webserver_interface=127.15.75.190
--webserver_port=45965
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab
--principal=kudu/127.15.75.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:42113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.75.190:35621
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:18:34.793906 5105 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:18:34.794121 5105 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:18:34.794158 5105 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:18:34.797992 5105 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:18:34.798043 5105 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:18:34.798062 5105 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:18:34.798079 5105 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:18:34.798094 5105 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:18:34.802983 5105 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:42113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.15.75.190:35621
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab
--principal=kudu/127.15.75.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:35621
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.75.190
--webserver_port=45965
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5105
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:18:34.803805 5105 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:18:34.804672 5105 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:18:34.818717 5116 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:34.818768 5105 server_base.cc:1034] running on GCE node
W20260809 02:18:34.818711 5113 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:34.818711 5112 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:34.819674 5105 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:18:34.820822 5105 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:18:34.822031 5105 hybrid_clock.cc:648] HybridClock initialized: now 1786241914822007 us; error 37 us; skew 500 ppm
Aug 09 02:18:34 dist-test-slave-s11t krb5kdc[4258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241914, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:18:34.824934 5105 init.cc:377] Logged in from keytab as kudu/127.15.75.190@KRBTEST.COM (short username kudu)
I20260809 02:18:34.825766 5105 webserver.cc:460] Webserver started at http://127.15.75.190:45965/ using document root <none> and password file <none>
I20260809 02:18:34.826299 5105 fs_manager.cc:359] Metadata directory not provided
I20260809 02:18:34.826368 5105 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:18:34.830112 5105 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260809 02:18:34.832146 5125 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:34.833431 5105 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 02:18:34.833570 5105 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/master-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/master-0/wal
uuid: "84a663e56c0248d1afacc5b1b6bf0298"
format_stamp: "Formatted at 2026-08-09 02:18:33 on dist-test-slave-s11t"
server_key: "04e8b42f6479cf489e38a750b41650d7"
server_key_iv: "8a24440a2521a44c67a8ed3ab63a5a71"
server_key_version: "encryptionkey@0"
I20260809 02:18:34.834144 5105 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:18:34.850373 5105 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:18:34.853087 5105 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:18:34.853302 5105 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:18:34.860514 5105 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.190:35621
I20260809 02:18:34.860813 5198 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.190:35621 every 8 connection(s)
I20260809 02:18:34.861692 5105 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
I20260809 02:18:34.865965 5199 sys_catalog.cc:263] Verifying existing consensus state
I20260809 02:18:34.867519 5199 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84a663e56c0248d1afacc5b1b6bf0298: Bootstrap starting.
I20260809 02:18:34.871802 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 5105
I20260809 02:18:34.872316 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.129:42815
--local_ip_for_outbound_sockets=127.15.75.129
--tserver_master_addrs=127.15.75.190:35621
--webserver_port=40793
--webserver_interface=127.15.75.129
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab
--principal=kudu/127.15.75.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:42113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 02:18:34.885684 5199 log.cc:824] T 00000000000000000000000000000000 P 84a663e56c0248d1afacc5b1b6bf0298: Log is configured to *not* fsync() on all Append() calls
I20260809 02:18:34.894484 5199 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84a663e56c0248d1afacc5b1b6bf0298: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 02:18:34.894843 5199 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84a663e56c0248d1afacc5b1b6bf0298: Bootstrap complete.
I20260809 02:18:34.898936 5199 raft_consensus.cc:348] T 00000000000000000000000000000000 P 84a663e56c0248d1afacc5b1b6bf0298 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84a663e56c0248d1afacc5b1b6bf0298" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 35621 } }
I20260809 02:18:34.899299 5199 raft_consensus.cc:729] T 00000000000000000000000000000000 P 84a663e56c0248d1afacc5b1b6bf0298 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84a663e56c0248d1afacc5b1b6bf0298, State: Initialized, Role: FOLLOWER
I20260809 02:18:34.899760 5199 consensus_queue.cc:260] T 00000000000000000000000000000000 P 84a663e56c0248d1afacc5b1b6bf0298 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84a663e56c0248d1afacc5b1b6bf0298" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 35621 } }
I20260809 02:18:34.899868 5199 raft_consensus.cc:388] T 00000000000000000000000000000000 P 84a663e56c0248d1afacc5b1b6bf0298 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:18:34.899960 5199 raft_consensus.cc:482] T 00000000000000000000000000000000 P 84a663e56c0248d1afacc5b1b6bf0298 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:18:34.900064 5199 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 84a663e56c0248d1afacc5b1b6bf0298 [term 1 FOLLOWER]: Advancing to term 2
I20260809 02:18:34.901923 5199 raft_consensus.cc:504] T 00000000000000000000000000000000 P 84a663e56c0248d1afacc5b1b6bf0298 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84a663e56c0248d1afacc5b1b6bf0298" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 35621 } }
I20260809 02:18:34.902243 5199 leader_election.cc:302] T 00000000000000000000000000000000 P 84a663e56c0248d1afacc5b1b6bf0298 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 84a663e56c0248d1afacc5b1b6bf0298; no voters:
I20260809 02:18:34.902570 5199 leader_election.cc:288] T 00000000000000000000000000000000 P 84a663e56c0248d1afacc5b1b6bf0298 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 02:18:34.902654 5204 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 84a663e56c0248d1afacc5b1b6bf0298 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 02:18:34.902863 5204 raft_consensus.cc:686] T 00000000000000000000000000000000 P 84a663e56c0248d1afacc5b1b6bf0298 [term 2 LEADER]: Becoming Leader. State: Replica: 84a663e56c0248d1afacc5b1b6bf0298, State: Running, Role: LEADER
I20260809 02:18:34.903143 5204 consensus_queue.cc:237] T 00000000000000000000000000000000 P 84a663e56c0248d1afacc5b1b6bf0298 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84a663e56c0248d1afacc5b1b6bf0298" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 35621 } }
I20260809 02:18:34.903775 5199 sys_catalog.cc:564] T 00000000000000000000000000000000 P 84a663e56c0248d1afacc5b1b6bf0298 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:18:34.904022 5204 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84a663e56c0248d1afacc5b1b6bf0298 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 84a663e56c0248d1afacc5b1b6bf0298. Latest consensus state: current_term: 2 leader_uuid: "84a663e56c0248d1afacc5b1b6bf0298" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84a663e56c0248d1afacc5b1b6bf0298" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 35621 } } }
I20260809 02:18:34.904131 5204 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84a663e56c0248d1afacc5b1b6bf0298 [sys.catalog]: This master's current role is: LEADER
I20260809 02:18:34.904495 5212 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:18:34.904538 5208 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84a663e56c0248d1afacc5b1b6bf0298 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "84a663e56c0248d1afacc5b1b6bf0298" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84a663e56c0248d1afacc5b1b6bf0298" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 35621 } } }
I20260809 02:18:34.904783 5208 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84a663e56c0248d1afacc5b1b6bf0298 [sys.catalog]: This master's current role is: LEADER
I20260809 02:18:34.909227 5212 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:18:34.909842 5212 catalog_manager.cc:1259] Loaded cluster ID: 68aea2d3fcfe4278a84f4fdaa2e4afae
I20260809 02:18:34.909935 5212 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:18:34.911317 5212 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260809 02:18:34.916144 5212 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260809 02:18:34.921433 5212 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:18:34.922091 5212 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 84a663e56c0248d1afacc5b1b6bf0298: Loaded TSK: 0
I20260809 02:18:34.923239 5212 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 02:18:34.986310 5202 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:18:34.986565 5202 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:18:34.986635 5202 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:18:34.990826 5202 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:18:34.990916 5202 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:18:34.991003 5202 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.129
I20260809 02:18:34.996178 5202 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:42113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab
--principal=kudu/127.15.75.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.129:42815
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.75.129
--webserver_port=40793
--tserver_master_addrs=127.15.75.190:35621
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5202
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.129
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:18:34.997246 5202 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:18:34.998234 5202 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:18:35.007889 5244 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:35.007892 5240 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:35.007908 5239 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:35.008723 5202 server_base.cc:1034] running on GCE node
I20260809 02:18:35.009161 5202 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:18:35.009819 5202 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:18:35.011011 5202 hybrid_clock.cc:648] HybridClock initialized: now 1786241915010997 us; error 33 us; skew 500 ppm
Aug 09 02:18:35 dist-test-slave-s11t krb5kdc[4258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241915, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:18:35.013849 5202 init.cc:377] Logged in from keytab as kudu/127.15.75.129@KRBTEST.COM (short username kudu)
I20260809 02:18:35.014997 5202 webserver.cc:460] Webserver started at http://127.15.75.129:40793/ using document root <none> and password file <none>
I20260809 02:18:35.015548 5202 fs_manager.cc:359] Metadata directory not provided
I20260809 02:18:35.015625 5202 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:18:35.019520 5202 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260809 02:18:35.021761 5253 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:35.023037 5202 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 02:18:35.023131 5202 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-0/wal
uuid: "352b7fb566bf458585a84dd49f8c07dc"
format_stamp: "Formatted at 2026-08-09 02:18:33 on dist-test-slave-s11t"
server_key: "c40cb3aae85a366be328e883500d52f8"
server_key_iv: "6876e9e96b4ce7b076beac3486380a8b"
server_key_version: "encryptionkey@0"
I20260809 02:18:35.023695 5202 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:18:35.049453 5202 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:18:35.052778 5202 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:18:35.052983 5202 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:18:35.053599 5202 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:18:35.054657 5263 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 02:18:35.058542 5202 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 02:18:35.058599 5202 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.000s
I20260809 02:18:35.058668 5202 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 02:18:35.060276 5202 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 02:18:35.060322 5202 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260809 02:18:35.060604 5263 tablet_bootstrap.cc:492] T adc1ba9b16534bf7ad003c8946235627 P 352b7fb566bf458585a84dd49f8c07dc: Bootstrap starting.
I20260809 02:18:35.071249 5202 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.129:42815
I20260809 02:18:35.071292 5424 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.129:42815 every 8 connection(s)
I20260809 02:18:35.072155 5202 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
I20260809 02:18:35.079996 5263 log.cc:824] T adc1ba9b16534bf7ad003c8946235627 P 352b7fb566bf458585a84dd49f8c07dc: Log is configured to *not* fsync() on all Append() calls
I20260809 02:18:35.080037 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 5202
I20260809 02:18:35.080565 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.130:46695
--local_ip_for_outbound_sockets=127.15.75.130
--tserver_master_addrs=127.15.75.190:35621
--webserver_port=42913
--webserver_interface=127.15.75.130
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab
--principal=kudu/127.15.75.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:42113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 02:18:35 dist-test-slave-s11t krb5kdc[4258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241915, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.129@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:18:35.088429 5263 tablet_bootstrap.cc:492] T adc1ba9b16534bf7ad003c8946235627 P 352b7fb566bf458585a84dd49f8c07dc: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 02:18:35.088881 5263 tablet_bootstrap.cc:492] T adc1ba9b16534bf7ad003c8946235627 P 352b7fb566bf458585a84dd49f8c07dc: Bootstrap complete.
I20260809 02:18:35.090437 5263 ts_tablet_manager.cc:1397] T adc1ba9b16534bf7ad003c8946235627 P 352b7fb566bf458585a84dd49f8c07dc: Time spent bootstrapping tablet: real 0.030s user 0.024s sys 0.005s
I20260809 02:18:35.090672 5430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:35.074142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.129:40817 (local address 127.15.75.190:35621)
0809 02:18:35.074552 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:35.074571 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:35.074974 (+ 403us) server_negotiation.cc:407] Connection header received
0809 02:18:35.076288 (+ 1314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:35.076298 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:35.076574 (+ 276us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:35.076895 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:35.078401 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:35.079349 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:35.080940 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:35.081621 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:35.081787 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:35.084438 (+ 2651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:35.084445 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:35.084460 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:18:35.084491 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:35.086659 (+ 2168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:35.087356 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:35.087361 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:35.087366 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:35.087513 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:35.087986 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:35.087990 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:35.087991 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:35.088423 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:35.088564 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:35.089385 (+ 821us) server_negotiation.cc:299] Negotiation successful
0809 02:18:35.089694 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":141,"threads_started":1}
I20260809 02:18:35.090672 5432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:35.074110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:35621 (local address 127.15.75.129:40817)
0809 02:18:35.074750 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:35.074824 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:35.075887 (+ 1063us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:35.077021 (+ 1134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:35.077029 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:35.077494 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:18:35.078228 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:35.078248 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:35.079518 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:35.079523 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:35.080761 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:35.080776 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:35.081776 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:35.081783 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:35.082035 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:35.082733 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:18:35.082751 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:18:35.084221 (+ 1470us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:18:35.086794 (+ 2573us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:18:35.086802 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:18:35.086819 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:18:35.087220 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:18:35.087630 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:18:35.087634 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:18:35.087638 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:18:35.087847 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:18:35.088635 (+ 788us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:18:35.088644 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:18:35.089021 (+ 377us) client_negotiation.cc:769] Sending connection context
0809 02:18:35.089254 (+ 233us) client_negotiation.cc:240] Negotiation successful
0809 02:18:35.089617 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":160,"threads_started":1}
I20260809 02:18:35.093106 5263 raft_consensus.cc:348] T adc1ba9b16534bf7ad003c8946235627 P 352b7fb566bf458585a84dd49f8c07dc [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d295396cdcae4b57acab477dfe1bd61d" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 46695 } } peers { permanent_uuid: "36d5cac71f2a4ef9bbfee682d6c3f712" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 46171 } } peers { permanent_uuid: "352b7fb566bf458585a84dd49f8c07dc" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 42815 } }
I20260809 02:18:35.093624 5263 raft_consensus.cc:729] T adc1ba9b16534bf7ad003c8946235627 P 352b7fb566bf458585a84dd49f8c07dc [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 352b7fb566bf458585a84dd49f8c07dc, State: Initialized, Role: FOLLOWER
I20260809 02:18:35.093731 5428 heartbeater.cc:347] Connected to a master server at 127.15.75.190:35621
I20260809 02:18:35.093940 5428 heartbeater.cc:464] Registering TS with master...
I20260809 02:18:35.094131 5263 consensus_queue.cc:260] T adc1ba9b16534bf7ad003c8946235627 P 352b7fb566bf458585a84dd49f8c07dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d295396cdcae4b57acab477dfe1bd61d" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 46695 } } peers { permanent_uuid: "36d5cac71f2a4ef9bbfee682d6c3f712" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 46171 } } peers { permanent_uuid: "352b7fb566bf458585a84dd49f8c07dc" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 42815 } }
I20260809 02:18:35.094455 5428 heartbeater.cc:511] Master 127.15.75.190:35621 requested a full tablet report, sending...
I20260809 02:18:35.095070 5263 ts_tablet_manager.cc:1428] T adc1ba9b16534bf7ad003c8946235627 P 352b7fb566bf458585a84dd49f8c07dc: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260809 02:18:35.096833 5149 ts_manager.cc:194] Registered new tserver with Master: 352b7fb566bf458585a84dd49f8c07dc (127.15.75.129:42815)
I20260809 02:18:35.099939 5149 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.75.129@KRBTEST.COM'} at 127.15.75.129:40817
I20260809 02:18:35.102444 5428 heartbeater.cc:503] Master 127.15.75.190:35621 was elected leader, sending a full tablet report...
Aug 09 02:18:35 dist-test-slave-s11t krb5kdc[4258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241914, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.190@KRBTEST.COM for kudu/127.15.75.129@KRBTEST.COM
I20260809 02:18:35.110829 5437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:35.099374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.129:42815 (local address 127.0.0.1:40878)
0809 02:18:35.100045 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:35.100085 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:35.100217 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:35.100725 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:35.100732 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:35.101104 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:18:35.101198 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:35.101204 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:35.102232 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:35.102237 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:35.102988 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:35.102999 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:35.104428 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:35.104435 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:35.105196 (+ 761us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:35.105704 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:18:35.105716 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:18:35.107321 (+ 1605us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:18:35.109145 (+ 1824us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:18:35.109155 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:18:35.109160 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:18:35.109379 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:18:35.109688 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:18:35.109691 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:18:35.109693 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:18:35.109775 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:18:35.110231 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:18:35.110237 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:18:35.110348 (+ 111us) client_negotiation.cc:769] Sending connection context
0809 02:18:35.110483 (+ 135us) client_negotiation.cc:240] Negotiation successful
0809 02:18:35.110660 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":104,"threads_started":1}
I20260809 02:18:35.110857 5438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:35.099466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40878 (local address 127.15.75.129:42815)
0809 02:18:35.099801 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:35.099811 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:35.100190 (+ 379us) server_negotiation.cc:407] Connection header received
0809 02:18:35.100361 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:35.100367 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:35.100550 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:35.100710 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:35.101318 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:35.102122 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:35.103832 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:35.104311 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:35.104473 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:35.107457 (+ 2984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:35.107466 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:35.107478 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:18:35.107509 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:35.109039 (+ 1530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:35.109504 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:35.109510 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:35.109513 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:35.109593 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:35.109870 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:35.109874 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:35.109876 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:35.110105 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:35.110239 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:35.110562 (+ 323us) server_negotiation.cc:299] Negotiation successful
0809 02:18:35.110713 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":105,"threads_started":1}
I20260809 02:18:35.111747 5323 tablet_service.cc:1514] Processing DeleteTablet for tablet adc1ba9b16534bf7ad003c8946235627 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:18:34 UTC) from {username='kudu', principal='kudu/127.15.75.190@KRBTEST.COM'} at 127.0.0.1:40878
I20260809 02:18:35.112628 5440 tablet_replica.cc:330] stopping tablet replica
I20260809 02:18:35.112911 5440 raft_consensus.cc:2229] T adc1ba9b16534bf7ad003c8946235627 P 352b7fb566bf458585a84dd49f8c07dc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:18:35.113152 5440 raft_consensus.cc:2258] T adc1ba9b16534bf7ad003c8946235627 P 352b7fb566bf458585a84dd49f8c07dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:18:35.113962 5440 ts_tablet_manager.cc:1905] T adc1ba9b16534bf7ad003c8946235627 P 352b7fb566bf458585a84dd49f8c07dc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:18:35.117198 5440 ts_tablet_manager.cc:1918] T adc1ba9b16534bf7ad003c8946235627 P 352b7fb566bf458585a84dd49f8c07dc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:18:35.117307 5440 log.cc:1196] T adc1ba9b16534bf7ad003c8946235627 P 352b7fb566bf458585a84dd49f8c07dc: Deleting WAL directory at /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-0/wal/wals/adc1ba9b16534bf7ad003c8946235627
I20260809 02:18:35.117578 5440 ts_tablet_manager.cc:1939] T adc1ba9b16534bf7ad003c8946235627 P 352b7fb566bf458585a84dd49f8c07dc: Deleting consensus metadata
I20260809 02:18:35.118996 5132 catalog_manager.cc:4841] TS 352b7fb566bf458585a84dd49f8c07dc (127.15.75.129:42815): tablet adc1ba9b16534bf7ad003c8946235627 (table test-table [id=d36b6bd551164a73ab85b5e95fe4557b]) successfully deleted
W20260809 02:18:35.198252 5435 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:18:35.198483 5435 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:18:35.198551 5435 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:18:35.202438 5435 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:18:35.202487 5435 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:18:35.202580 5435 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.130
I20260809 02:18:35.208056 5435 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:42113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab
--principal=kudu/127.15.75.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.130:46695
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.75.130
--webserver_port=42913
--tserver_master_addrs=127.15.75.190:35621
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5435
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.130
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:18:35.208956 5435 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:18:35.209895 5435 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:18:35.219152 5448 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:35.219262 5435 server_base.cc:1034] running on GCE node
W20260809 02:18:35.219165 5451 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:35.219165 5447 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:35.219771 5435 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:18:35.220386 5435 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:18:35.221570 5435 hybrid_clock.cc:648] HybridClock initialized: now 1786241915221528 us; error 54 us; skew 500 ppm
Aug 09 02:18:35 dist-test-slave-s11t krb5kdc[4258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241915, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:18:35.224253 5435 init.cc:377] Logged in from keytab as kudu/127.15.75.130@KRBTEST.COM (short username kudu)
I20260809 02:18:35.225099 5435 webserver.cc:460] Webserver started at http://127.15.75.130:42913/ using document root <none> and password file <none>
I20260809 02:18:35.225579 5435 fs_manager.cc:359] Metadata directory not provided
I20260809 02:18:35.225642 5435 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:18:35.229421 5435 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 02:18:35.231345 5464 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:35.232501 5435 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 02:18:35.232590 5435 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-1/wal
uuid: "d295396cdcae4b57acab477dfe1bd61d"
format_stamp: "Formatted at 2026-08-09 02:18:34 on dist-test-slave-s11t"
server_key: "f2f70264978910559b7a16b1d94caa58"
server_key_iv: "35a31c73d12962326cff4ca00a57858e"
server_key_version: "encryptionkey@0"
I20260809 02:18:35.233107 5435 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:18:35.256495 5435 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:18:35.259552 5435 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:18:35.259692 5435 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:18:35.260174 5435 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:18:35.260913 5435 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:18:35.260959 5435 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:35.260998 5435 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:18:35.261018 5435 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:35.269791 5435 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.130:46695
I20260809 02:18:35.269822 5636 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.130:46695 every 8 connection(s)
I20260809 02:18:35.270711 5435 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
I20260809 02:18:35.279243 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 5435
I20260809 02:18:35.279667 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.131:46171
--local_ip_for_outbound_sockets=127.15.75.131
--tserver_master_addrs=127.15.75.190:35621
--webserver_port=34579
--webserver_interface=127.15.75.131
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab
--principal=kudu/127.15.75.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:42113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 02:18:35 dist-test-slave-s11t krb5kdc[4258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241915, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.130@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:18:35.284386 5430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:35.272402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.130:51309 (local address 127.15.75.190:35621)
0809 02:18:35.272582 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:35.272587 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:35.273439 (+ 852us) server_negotiation.cc:407] Connection header received
0809 02:18:35.274750 (+ 1311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:35.274753 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:35.274801 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:35.274877 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:35.275951 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:35.276488 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:35.277623 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:35.277863 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:35.277945 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:35.280448 (+ 2503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:35.280452 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:35.280454 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:18:35.280479 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:35.282272 (+ 1793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:35.282781 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:35.282787 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:35.282790 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:35.282915 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:35.283253 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:35.283256 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:35.283258 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:35.283484 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:35.283563 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:35.284155 (+ 592us) server_negotiation.cc:299] Negotiation successful
0809 02:18:35.284265 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 02:18:35.285591 5641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:35.272843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:35621 (local address 127.15.75.130:51309)
0809 02:18:35.273279 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:35.273319 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:35.274531 (+ 1212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:35.275046 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:35.275051 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:35.275381 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:18:35.275811 (+ 430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:35.275824 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:35.276594 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:35.276596 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:35.277507 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:35.277516 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:35.277992 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:35.277994 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:35.278129 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:35.278624 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:18:35.278640 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:18:35.280264 (+ 1624us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:18:35.282402 (+ 2138us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:18:35.282407 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:18:35.282417 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:18:35.282660 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:18:35.283025 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:18:35.283027 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:18:35.283028 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:18:35.283151 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:18:35.283596 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:18:35.283605 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:18:35.283910 (+ 305us) client_negotiation.cc:769] Sending connection context
0809 02:18:35.284117 (+ 207us) client_negotiation.cc:240] Negotiation successful
0809 02:18:35.284467 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":125,"threads_started":1}
I20260809 02:18:35.286810 5638 heartbeater.cc:347] Connected to a master server at 127.15.75.190:35621
I20260809 02:18:35.287060 5638 heartbeater.cc:464] Registering TS with master...
I20260809 02:18:35.287519 5638 heartbeater.cc:511] Master 127.15.75.190:35621 requested a full tablet report, sending...
I20260809 02:18:35.288592 5149 ts_manager.cc:194] Registered new tserver with Master: d295396cdcae4b57acab477dfe1bd61d (127.15.75.130:46695)
I20260809 02:18:35.289533 5149 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.75.130@KRBTEST.COM'} at 127.15.75.130:51309
W20260809 02:18:35.391093 5648 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:18:35.391360 5648 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:18:35.391407 5648 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:18:35.395309 5648 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:18:35.395360 5648 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:18:35.395414 5648 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.131
I20260809 02:18:35.400363 5648 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:42113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab
--principal=kudu/127.15.75.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.131:46171
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.75.131
--webserver_port=34579
--tserver_master_addrs=127.15.75.190:35621
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5648
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.131
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:18:35.401264 5648 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:18:35.402181 5648 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:18:35.411809 5661 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:35.411809 5657 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:35.411902 5659 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:35.412086 5648 server_base.cc:1034] running on GCE node
I20260809 02:18:35.412669 5648 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:18:35.413354 5648 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:18:35.414518 5648 hybrid_clock.cc:648] HybridClock initialized: now 1786241915414492 us; error 37 us; skew 500 ppm
Aug 09 02:18:35 dist-test-slave-s11t krb5kdc[4258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241915, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:18:35.417376 5648 init.cc:377] Logged in from keytab as kudu/127.15.75.131@KRBTEST.COM (short username kudu)
I20260809 02:18:35.418145 5648 webserver.cc:460] Webserver started at http://127.15.75.131:34579/ using document root <none> and password file <none>
I20260809 02:18:35.418716 5648 fs_manager.cc:359] Metadata directory not provided
I20260809 02:18:35.418813 5648 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:18:35.422619 5648 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.004s
I20260809 02:18:35.424654 5671 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:35.425880 5648 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 02:18:35.426009 5648 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-2/wal
uuid: "36d5cac71f2a4ef9bbfee682d6c3f712"
format_stamp: "Formatted at 2026-08-09 02:18:34 on dist-test-slave-s11t"
server_key: "417270fd0d930863d79dfbe3d92c882d"
server_key_iv: "88efa4f9f7d188aef74706e9b6e19694"
server_key_version: "encryptionkey@0"
I20260809 02:18:35.426524 5648 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:18:35.442430 5648 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:18:35.445122 5648 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:18:35.445323 5648 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:18:35.445884 5648 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:18:35.446676 5648 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:18:35.446722 5648 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:35.446792 5648 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:18:35.446830 5648 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:35.455447 5648 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.131:46171
I20260809 02:18:35.455480 5851 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.131:46171 every 8 connection(s)
I20260809 02:18:35.456383 5648 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
I20260809 02:18:35.457772 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 5648
Aug 09 02:18:35 dist-test-slave-s11t krb5kdc[4258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241915, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.131@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:18:35.467521 5857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:35.458835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42318 (local address 127.15.75.190:35621)
0809 02:18:35.459116 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:35.459120 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:35.459130 (+ 10us) server_negotiation.cc:407] Connection header received
0809 02:18:35.459177 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:35.459179 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:35.459220 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:35.459375 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:35.460141 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:35.460838 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:35.461709 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:35.462068 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:35.462176 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:35.463568 (+ 1392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:35.463575 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:35.463579 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:18:35.463609 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:35.465536 (+ 1927us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:35.466047 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:35.466050 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:35.466051 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:35.466246 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:35.466596 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:35.466599 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:35.466600 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:35.466788 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:35.467037 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:35.467202 (+ 165us) server_negotiation.cc:299] Negotiation successful
0809 02:18:35.467332 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":97,"threads_started":1}
I20260809 02:18:35.470481 5430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:35.458235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.131:55265 (local address 127.15.75.190:35621)
0809 02:18:35.458374 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:35.458379 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:35.459304 (+ 925us) server_negotiation.cc:407] Connection header received
0809 02:18:35.460625 (+ 1321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:35.460628 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:35.460675 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:35.460787 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:35.461803 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:35.462437 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:35.463599 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:35.463817 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:35.463948 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:35.466799 (+ 2851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:35.466802 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:35.466805 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:18:35.466832 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:35.468572 (+ 1740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:35.469093 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:35.469098 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:35.469100 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:35.469185 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:35.469611 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:35.469614 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:35.469615 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:35.469772 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:35.469848 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:35.470289 (+ 441us) server_negotiation.cc:299] Negotiation successful
0809 02:18:35.470375 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 02:18:35.471280 5856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:35.458669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:35621 (local address 127.15.75.131:55265)
0809 02:18:35.459148 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:35.459197 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:35.460439 (+ 1242us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:35.460913 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:35.460919 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:35.461238 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:18:35.461681 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:35.461694 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:35.462565 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:35.462569 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:35.463457 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:35.463494 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:35.463984 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:35.463988 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:35.464214 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:35.464739 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:18:35.464759 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:18:35.466650 (+ 1891us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:18:35.468702 (+ 2052us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:18:35.468710 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:18:35.468724 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:18:35.468988 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:18:35.469305 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:18:35.469308 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:18:35.469312 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:18:35.469518 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:18:35.469871 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:18:35.469876 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:18:35.470088 (+ 212us) client_negotiation.cc:769] Sending connection context
0809 02:18:35.470243 (+ 155us) client_negotiation.cc:240] Negotiation successful
0809 02:18:35.470472 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":141,"threads_started":1}
I20260809 02:18:35.472317 5852 heartbeater.cc:347] Connected to a master server at 127.15.75.190:35621
I20260809 02:18:35.472560 5852 heartbeater.cc:464] Registering TS with master...
I20260809 02:18:35.473049 5852 heartbeater.cc:511] Master 127.15.75.190:35621 requested a full tablet report, sending...
I20260809 02:18:35.473996 5149 ts_manager.cc:194] Registered new tserver with Master: 36d5cac71f2a4ef9bbfee682d6c3f712 (127.15.75.131:46171)
I20260809 02:18:35.474545 5149 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.75.131@KRBTEST.COM'} at 127.15.75.131:55265
I20260809 02:18:35.481062 15662 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 02:18:35.484871 5430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:35.481595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42334 (local address 127.15.75.190:35621)
0809 02:18:35.481710 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:35.481713 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:35.481908 (+ 195us) server_negotiation.cc:407] Connection header received
0809 02:18:35.482030 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:35.482032 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:35.482111 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:35.482188 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:18:35.482374 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:35.482868 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:35.483665 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:35.483877 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:35.483960 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:35.484215 (+ 255us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:18:35.484529 (+ 314us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:18:35.484614 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:35.484670 (+ 56us) server_negotiation.cc:299] Negotiation successful
0809 02:18:35.484774 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 02:18:35.485423 5149 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:42334:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:18:35.488526 5149 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:18:35.496264 5323 tablet_service.cc:1467] Processing CreateTablet for tablet c86bd422f6054e4291f4a17d3f6cefb6 (DEFAULT_TABLE table=test-table [id=6b6cf70f0d484274b68aeac025d5fde2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:18:35.496953 5323 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c86bd422f6054e4291f4a17d3f6cefb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:18:35.499826 5263 tablet_bootstrap.cc:492] T c86bd422f6054e4291f4a17d3f6cefb6 P 352b7fb566bf458585a84dd49f8c07dc: Bootstrap starting.
I20260809 02:18:35.500308 5437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:35.495168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.130:46695 (local address 127.0.0.1:35822)
0809 02:18:35.495373 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:35.495389 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:35.495466 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:35.496435 (+ 969us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:35.496437 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:35.496454 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:18:35.496535 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:35.496557 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:35.498081 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:35.498087 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:35.498899 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:35.498909 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:35.499889 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:35.499892 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:35.500000 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:35.500049 (+ 49us) client_negotiation.cc:769] Sending connection context
0809 02:18:35.500105 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 02:18:35.500190 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142}
I20260809 02:18:35.500660 5263 tablet_bootstrap.cc:654] T c86bd422f6054e4291f4a17d3f6cefb6 P 352b7fb566bf458585a84dd49f8c07dc: Neither blocks nor log segments found. Creating new log.
I20260809 02:18:35.501206 5863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:35.495414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35822 (local address 127.15.75.130:46695)
0809 02:18:35.495903 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:35.495912 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:35.495932 (+ 20us) server_negotiation.cc:407] Connection header received
0809 02:18:35.496034 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:35.496039 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:35.496285 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:35.496453 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:18:35.496721 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:35.497928 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:35.499036 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:35.499757 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:35.500766 (+ 1009us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:35.500908 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:35.500959 (+ 51us) server_negotiation.cc:299] Negotiation successful
0809 02:18:35.501053 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":146,"threads_started":1}
I20260809 02:18:35.501958 5864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:35.495507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.131:46171 (local address 127.0.0.1:60710)
0809 02:18:35.496163 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:35.496177 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:35.496274 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:35.496756 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:35.496761 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:35.496782 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:18:35.496875 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:35.496881 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:35.498180 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:35.498183 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:35.499302 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:35.499315 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:35.501573 (+ 2258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:35.501577 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:35.501666 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:35.501688 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 02:18:35.501769 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 02:18:35.501834 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":596,"thread_start_us":117,"threads_started":1}
I20260809 02:18:35.502099 5263 tablet_bootstrap.cc:492] T c86bd422f6054e4291f4a17d3f6cefb6 P 352b7fb566bf458585a84dd49f8c07dc: No bootstrap required, opened a new log
I20260809 02:18:35.502187 5263 ts_tablet_manager.cc:1397] T c86bd422f6054e4291f4a17d3f6cefb6 P 352b7fb566bf458585a84dd49f8c07dc: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 02:18:35.502223 5866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:35.495768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60710 (local address 127.15.75.131:46171)
0809 02:18:35.496134 (+ 366us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:35.496148 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:35.496268 (+ 120us) server_negotiation.cc:407] Connection header received
0809 02:18:35.496389 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:35.496397 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:35.496567 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:35.497028 (+ 461us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:18:35.497192 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:35.498028 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:35.499462 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:35.501246 (+ 1784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:35.501360 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:35.501507 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:35.501807 (+ 300us) server_negotiation.cc:299] Negotiation successful
0809 02:18:35.501915 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":160,"threads_started":1}
I20260809 02:18:35.502604 5263 raft_consensus.cc:348] T c86bd422f6054e4291f4a17d3f6cefb6 P 352b7fb566bf458585a84dd49f8c07dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d295396cdcae4b57acab477dfe1bd61d" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 46695 } } peers { permanent_uuid: "36d5cac71f2a4ef9bbfee682d6c3f712" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 46171 } } peers { permanent_uuid: "352b7fb566bf458585a84dd49f8c07dc" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 42815 } }
I20260809 02:18:35.502712 5263 raft_consensus.cc:374] T c86bd422f6054e4291f4a17d3f6cefb6 P 352b7fb566bf458585a84dd49f8c07dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:18:35.502740 5263 raft_consensus.cc:729] T c86bd422f6054e4291f4a17d3f6cefb6 P 352b7fb566bf458585a84dd49f8c07dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 352b7fb566bf458585a84dd49f8c07dc, State: Initialized, Role: FOLLOWER
I20260809 02:18:35.502868 5263 consensus_queue.cc:260] T c86bd422f6054e4291f4a17d3f6cefb6 P 352b7fb566bf458585a84dd49f8c07dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d295396cdcae4b57acab477dfe1bd61d" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 46695 } } peers { permanent_uuid: "36d5cac71f2a4ef9bbfee682d6c3f712" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 46171 } } peers { permanent_uuid: "352b7fb566bf458585a84dd49f8c07dc" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 42815 } }
I20260809 02:18:35.503158 5263 ts_tablet_manager.cc:1428] T c86bd422f6054e4291f4a17d3f6cefb6 P 352b7fb566bf458585a84dd49f8c07dc: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 02:18:35.505198 5530 tablet_service.cc:1467] Processing CreateTablet for tablet c86bd422f6054e4291f4a17d3f6cefb6 (DEFAULT_TABLE table=test-table [id=6b6cf70f0d484274b68aeac025d5fde2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:18:35.506083 5530 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c86bd422f6054e4291f4a17d3f6cefb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:18:35.511255 5749 tablet_service.cc:1467] Processing CreateTablet for tablet c86bd422f6054e4291f4a17d3f6cefb6 (DEFAULT_TABLE table=test-table [id=6b6cf70f0d484274b68aeac025d5fde2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:18:35.512151 5749 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c86bd422f6054e4291f4a17d3f6cefb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:18:35.513939 5868 tablet_bootstrap.cc:492] T c86bd422f6054e4291f4a17d3f6cefb6 P d295396cdcae4b57acab477dfe1bd61d: Bootstrap starting.
I20260809 02:18:35.516479 5868 tablet_bootstrap.cc:654] T c86bd422f6054e4291f4a17d3f6cefb6 P d295396cdcae4b57acab477dfe1bd61d: Neither blocks nor log segments found. Creating new log.
I20260809 02:18:35.517307 5868 log.cc:824] T c86bd422f6054e4291f4a17d3f6cefb6 P d295396cdcae4b57acab477dfe1bd61d: Log is configured to *not* fsync() on all Append() calls
I20260809 02:18:35.518622 5870 tablet_bootstrap.cc:492] T c86bd422f6054e4291f4a17d3f6cefb6 P 36d5cac71f2a4ef9bbfee682d6c3f712: Bootstrap starting.
I20260809 02:18:35.519356 5868 tablet_bootstrap.cc:492] T c86bd422f6054e4291f4a17d3f6cefb6 P d295396cdcae4b57acab477dfe1bd61d: No bootstrap required, opened a new log
I20260809 02:18:35.519557 5868 ts_tablet_manager.cc:1397] T c86bd422f6054e4291f4a17d3f6cefb6 P d295396cdcae4b57acab477dfe1bd61d: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 02:18:35.521494 5870 tablet_bootstrap.cc:654] T c86bd422f6054e4291f4a17d3f6cefb6 P 36d5cac71f2a4ef9bbfee682d6c3f712: Neither blocks nor log segments found. Creating new log.
I20260809 02:18:35.522310 5870 log.cc:824] T c86bd422f6054e4291f4a17d3f6cefb6 P 36d5cac71f2a4ef9bbfee682d6c3f712: Log is configured to *not* fsync() on all Append() calls
I20260809 02:18:35.523548 5868 raft_consensus.cc:348] T c86bd422f6054e4291f4a17d3f6cefb6 P d295396cdcae4b57acab477dfe1bd61d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d295396cdcae4b57acab477dfe1bd61d" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 46695 } } peers { permanent_uuid: "36d5cac71f2a4ef9bbfee682d6c3f712" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 46171 } } peers { permanent_uuid: "352b7fb566bf458585a84dd49f8c07dc" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 42815 } }
I20260809 02:18:35.523809 5868 raft_consensus.cc:374] T c86bd422f6054e4291f4a17d3f6cefb6 P d295396cdcae4b57acab477dfe1bd61d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:18:35.523861 5868 raft_consensus.cc:729] T c86bd422f6054e4291f4a17d3f6cefb6 P d295396cdcae4b57acab477dfe1bd61d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d295396cdcae4b57acab477dfe1bd61d, State: Initialized, Role: FOLLOWER
I20260809 02:18:35.524256 5870 tablet_bootstrap.cc:492] T c86bd422f6054e4291f4a17d3f6cefb6 P 36d5cac71f2a4ef9bbfee682d6c3f712: No bootstrap required, opened a new log
I20260809 02:18:35.524312 5868 consensus_queue.cc:260] T c86bd422f6054e4291f4a17d3f6cefb6 P d295396cdcae4b57acab477dfe1bd61d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d295396cdcae4b57acab477dfe1bd61d" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 46695 } } peers { permanent_uuid: "36d5cac71f2a4ef9bbfee682d6c3f712" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 46171 } } peers { permanent_uuid: "352b7fb566bf458585a84dd49f8c07dc" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 42815 } }
I20260809 02:18:35.524447 5870 ts_tablet_manager.cc:1397] T c86bd422f6054e4291f4a17d3f6cefb6 P 36d5cac71f2a4ef9bbfee682d6c3f712: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 02:18:35.525003 5638 heartbeater.cc:503] Master 127.15.75.190:35621 was elected leader, sending a full tablet report...
I20260809 02:18:35.525305 5868 ts_tablet_manager.cc:1428] T c86bd422f6054e4291f4a17d3f6cefb6 P d295396cdcae4b57acab477dfe1bd61d: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260809 02:18:35.527860 5870 raft_consensus.cc:348] T c86bd422f6054e4291f4a17d3f6cefb6 P 36d5cac71f2a4ef9bbfee682d6c3f712 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d295396cdcae4b57acab477dfe1bd61d" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 46695 } } peers { permanent_uuid: "36d5cac71f2a4ef9bbfee682d6c3f712" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 46171 } } peers { permanent_uuid: "352b7fb566bf458585a84dd49f8c07dc" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 42815 } }
I20260809 02:18:35.528076 5870 raft_consensus.cc:374] T c86bd422f6054e4291f4a17d3f6cefb6 P 36d5cac71f2a4ef9bbfee682d6c3f712 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:18:35.528158 5870 raft_consensus.cc:729] T c86bd422f6054e4291f4a17d3f6cefb6 P 36d5cac71f2a4ef9bbfee682d6c3f712 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36d5cac71f2a4ef9bbfee682d6c3f712, State: Initialized, Role: FOLLOWER
I20260809 02:18:35.528602 5870 consensus_queue.cc:260] T c86bd422f6054e4291f4a17d3f6cefb6 P 36d5cac71f2a4ef9bbfee682d6c3f712 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d295396cdcae4b57acab477dfe1bd61d" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 46695 } } peers { permanent_uuid: "36d5cac71f2a4ef9bbfee682d6c3f712" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 46171 } } peers { permanent_uuid: "352b7fb566bf458585a84dd49f8c07dc" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 42815 } }
I20260809 02:18:35.529254 5852 heartbeater.cc:503] Master 127.15.75.190:35621 was elected leader, sending a full tablet report...
I20260809 02:18:35.529562 5870 ts_tablet_manager.cc:1428] T c86bd422f6054e4291f4a17d3f6cefb6 P 36d5cac71f2a4ef9bbfee682d6c3f712: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260809 02:18:35.536406 5436 raft_consensus.cc:482] T c86bd422f6054e4291f4a17d3f6cefb6 P 352b7fb566bf458585a84dd49f8c07dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:18:35.536602 5436 raft_consensus.cc:504] T c86bd422f6054e4291f4a17d3f6cefb6 P 352b7fb566bf458585a84dd49f8c07dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d295396cdcae4b57acab477dfe1bd61d" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 46695 } } peers { permanent_uuid: "36d5cac71f2a4ef9bbfee682d6c3f712" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 46171 } } peers { permanent_uuid: "352b7fb566bf458585a84dd49f8c07dc" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 42815 } }
I20260809 02:18:35.537724 5436 leader_election.cc:288] T c86bd422f6054e4291f4a17d3f6cefb6 P 352b7fb566bf458585a84dd49f8c07dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d295396cdcae4b57acab477dfe1bd61d (127.15.75.130:46695), 36d5cac71f2a4ef9bbfee682d6c3f712 (127.15.75.131:46171)
I20260809 02:18:35.541771 5863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:35.537984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.129:39285 (local address 127.15.75.130:46695)
0809 02:18:35.538132 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:35.538138 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:35.538153 (+ 15us) server_negotiation.cc:407] Connection header received
0809 02:18:35.538295 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:35.538298 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:35.538362 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:35.538433 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:18:35.538842 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:35.539477 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:35.540668 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:35.541160 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:35.541252 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:35.541290 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:35.541575 (+ 285us) server_negotiation.cc:299] Negotiation successful
0809 02:18:35.541652 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 02:18:35.541796 5432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:35.537916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.130:46695 (local address 127.15.75.129:39285)
0809 02:18:35.538049 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:35.538071 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:35.538176 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:35.538545 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:35.538548 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:35.538567 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:18:35.538708 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:35.538718 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:35.539583 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:35.539587 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:35.540543 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:35.540553 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:35.541277 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:35.541281 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:35.541415 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:35.541464 (+ 49us) client_negotiation.cc:769] Sending connection context
0809 02:18:35.541542 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 02:18:35.541664 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48,"mutex_wait_us":25}
I20260809 02:18:35.541795 5866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:35.538277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.129:47869 (local address 127.15.75.131:46171)
0809 02:18:35.538394 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:35.538400 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:35.538648 (+ 248us) server_negotiation.cc:407] Connection header received
0809 02:18:35.538790 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:35.538793 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:35.538827 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:35.538887 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:18:35.539169 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:35.539694 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:35.540837 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:35.541265 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:35.541382 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:35.541430 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:35.541614 (+ 184us) server_negotiation.cc:299] Negotiation successful
0809 02:18:35.541685 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 02:18:35.542218 5878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:35.538194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.131:46171 (local address 127.15.75.129:47869)
0809 02:18:35.538556 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:35.538568 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:35.538662 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:35.538922 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:35.538925 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:35.538944 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:18:35.539065 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:35.539072 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:35.539799 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:35.539802 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:35.540723 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:35.540734 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:35.541395 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:35.541398 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:35.541474 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:35.541491 (+ 17us) client_negotiation.cc:769] Sending connection context
0809 02:18:35.541588 (+ 97us) client_negotiation.cc:240] Negotiation successful
0809 02:18:35.541678 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"mutex_wait_us":1,"thread_start_us":79,"threads_started":1}
I20260809 02:18:35.542385 5569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c86bd422f6054e4291f4a17d3f6cefb6" candidate_uuid: "352b7fb566bf458585a84dd49f8c07dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d295396cdcae4b57acab477dfe1bd61d" is_pre_election: true
I20260809 02:18:35.542668 5569 raft_consensus.cc:2454] T c86bd422f6054e4291f4a17d3f6cefb6 P d295396cdcae4b57acab477dfe1bd61d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 352b7fb566bf458585a84dd49f8c07dc in term 0.
I20260809 02:18:35.542750 5777 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c86bd422f6054e4291f4a17d3f6cefb6" candidate_uuid: "352b7fb566bf458585a84dd49f8c07dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d5cac71f2a4ef9bbfee682d6c3f712" is_pre_election: true
I20260809 02:18:35.543002 5777 raft_consensus.cc:2454] T c86bd422f6054e4291f4a17d3f6cefb6 P 36d5cac71f2a4ef9bbfee682d6c3f712 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 352b7fb566bf458585a84dd49f8c07dc in term 0.
I20260809 02:18:35.543138 5255 leader_election.cc:302] T c86bd422f6054e4291f4a17d3f6cefb6 P 352b7fb566bf458585a84dd49f8c07dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 352b7fb566bf458585a84dd49f8c07dc, d295396cdcae4b57acab477dfe1bd61d; no voters:
I20260809 02:18:35.543388 5436 raft_consensus.cc:2781] T c86bd422f6054e4291f4a17d3f6cefb6 P 352b7fb566bf458585a84dd49f8c07dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:18:35.543474 5436 raft_consensus.cc:482] T c86bd422f6054e4291f4a17d3f6cefb6 P 352b7fb566bf458585a84dd49f8c07dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:18:35.543536 5436 raft_consensus.cc:3037] T c86bd422f6054e4291f4a17d3f6cefb6 P 352b7fb566bf458585a84dd49f8c07dc [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:18:35.544492 5436 raft_consensus.cc:504] T c86bd422f6054e4291f4a17d3f6cefb6 P 352b7fb566bf458585a84dd49f8c07dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d295396cdcae4b57acab477dfe1bd61d" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 46695 } } peers { permanent_uuid: "36d5cac71f2a4ef9bbfee682d6c3f712" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 46171 } } peers { permanent_uuid: "352b7fb566bf458585a84dd49f8c07dc" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 42815 } }
I20260809 02:18:35.544843 5436 leader_election.cc:288] T c86bd422f6054e4291f4a17d3f6cefb6 P 352b7fb566bf458585a84dd49f8c07dc [CANDIDATE]: Term 1 election: Requested vote from peers d295396cdcae4b57acab477dfe1bd61d (127.15.75.130:46695), 36d5cac71f2a4ef9bbfee682d6c3f712 (127.15.75.131:46171)
I20260809 02:18:35.545199 5569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c86bd422f6054e4291f4a17d3f6cefb6" candidate_uuid: "352b7fb566bf458585a84dd49f8c07dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d295396cdcae4b57acab477dfe1bd61d"
I20260809 02:18:35.545316 5569 raft_consensus.cc:3037] T c86bd422f6054e4291f4a17d3f6cefb6 P d295396cdcae4b57acab477dfe1bd61d [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:18:35.545419 5777 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c86bd422f6054e4291f4a17d3f6cefb6" candidate_uuid: "352b7fb566bf458585a84dd49f8c07dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36d5cac71f2a4ef9bbfee682d6c3f712"
I20260809 02:18:35.545531 5777 raft_consensus.cc:3037] T c86bd422f6054e4291f4a17d3f6cefb6 P 36d5cac71f2a4ef9bbfee682d6c3f712 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:18:35.546602 5569 raft_consensus.cc:2454] T c86bd422f6054e4291f4a17d3f6cefb6 P d295396cdcae4b57acab477dfe1bd61d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 352b7fb566bf458585a84dd49f8c07dc in term 1.
I20260809 02:18:35.546751 5777 raft_consensus.cc:2454] T c86bd422f6054e4291f4a17d3f6cefb6 P 36d5cac71f2a4ef9bbfee682d6c3f712 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 352b7fb566bf458585a84dd49f8c07dc in term 1.
I20260809 02:18:35.546881 5255 leader_election.cc:302] T c86bd422f6054e4291f4a17d3f6cefb6 P 352b7fb566bf458585a84dd49f8c07dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 352b7fb566bf458585a84dd49f8c07dc, d295396cdcae4b57acab477dfe1bd61d; no voters:
I20260809 02:18:35.547036 5436 raft_consensus.cc:2781] T c86bd422f6054e4291f4a17d3f6cefb6 P 352b7fb566bf458585a84dd49f8c07dc [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:18:35.547240 5436 raft_consensus.cc:686] T c86bd422f6054e4291f4a17d3f6cefb6 P 352b7fb566bf458585a84dd49f8c07dc [term 1 LEADER]: Becoming Leader. State: Replica: 352b7fb566bf458585a84dd49f8c07dc, State: Running, Role: LEADER
I20260809 02:18:35.547492 5436 consensus_queue.cc:237] T c86bd422f6054e4291f4a17d3f6cefb6 P 352b7fb566bf458585a84dd49f8c07dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d295396cdcae4b57acab477dfe1bd61d" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 46695 } } peers { permanent_uuid: "36d5cac71f2a4ef9bbfee682d6c3f712" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 46171 } } peers { permanent_uuid: "352b7fb566bf458585a84dd49f8c07dc" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 42815 } }
I20260809 02:18:35.550261 5148 catalog_manager.cc:5495] T c86bd422f6054e4291f4a17d3f6cefb6 P 352b7fb566bf458585a84dd49f8c07dc reported cstate change: term changed from 0 to 1, leader changed from <none> to 352b7fb566bf458585a84dd49f8c07dc (127.15.75.129). New cstate: current_term: 1 leader_uuid: "352b7fb566bf458585a84dd49f8c07dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d295396cdcae4b57acab477dfe1bd61d" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 46695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36d5cac71f2a4ef9bbfee682d6c3f712" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 46171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "352b7fb566bf458585a84dd49f8c07dc" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 42815 } health_report { overall_health: HEALTHY } } }
I20260809 02:18:35.566931 5438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:35.563632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40880 (local address 127.15.75.129:42815)
0809 02:18:35.563768 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:35.563772 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:35.563785 (+ 13us) server_negotiation.cc:407] Connection header received
0809 02:18:35.563948 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:35.563950 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:35.563998 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:35.564077 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:18:35.564271 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:35.564942 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:35.565706 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:35.565928 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:35.566000 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:35.566221 (+ 221us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:18:35.566576 (+ 355us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:18:35.566656 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:35.566713 (+ 57us) server_negotiation.cc:299] Negotiation successful
0809 02:18:35.566755 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 02:18:35.570495 5777 raft_consensus.cc:1261] T c86bd422f6054e4291f4a17d3f6cefb6 P 36d5cac71f2a4ef9bbfee682d6c3f712 [term 1 FOLLOWER]: Refusing update from remote peer 352b7fb566bf458585a84dd49f8c07dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:18:35.570495 5569 raft_consensus.cc:1261] T c86bd422f6054e4291f4a17d3f6cefb6 P d295396cdcae4b57acab477dfe1bd61d [term 1 FOLLOWER]: Refusing update from remote peer 352b7fb566bf458585a84dd49f8c07dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:18:35.571190 5436 consensus_queue.cc:1035] T c86bd422f6054e4291f4a17d3f6cefb6 P 352b7fb566bf458585a84dd49f8c07dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "d295396cdcae4b57acab477dfe1bd61d" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 46695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:18:35.571417 5879 consensus_queue.cc:1035] T c86bd422f6054e4291f4a17d3f6cefb6 P 352b7fb566bf458585a84dd49f8c07dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "36d5cac71f2a4ef9bbfee682d6c3f712" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 46171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 02:18:35.573325 5429 tablet.cc:2369] T c86bd422f6054e4291f4a17d3f6cefb6 P 352b7fb566bf458585a84dd49f8c07dc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:18:35.579705 5882 mvcc.cc:205] Tried to move back new op lower bound from 7316446886170697728 to 7316446886084931584. Current Snapshot: MvccSnapshot[applied={T|T < 7316446886170697728}]
I20260809 02:18:35.581712 5884 mvcc.cc:205] Tried to move back new op lower bound from 7316446886170697728 to 7316446886084931584. Current Snapshot: MvccSnapshot[applied={T|T < 7316446886170697728}]
I20260809 02:18:35.586676 5148 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:42334:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:18:35.586848 5148 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:42334:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:18:35.589190 5148 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 84a663e56c0248d1afacc5b1b6bf0298: Sending DeleteTablet for 3 replicas of tablet c86bd422f6054e4291f4a17d3f6cefb6
I20260809 02:18:35.589871 5323 tablet_service.cc:1514] Processing DeleteTablet for tablet c86bd422f6054e4291f4a17d3f6cefb6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:18:35 UTC) from {username='kudu', principal='kudu/127.15.75.190@KRBTEST.COM'} at 127.0.0.1:40878
I20260809 02:18:35.589934 5530 tablet_service.cc:1514] Processing DeleteTablet for tablet c86bd422f6054e4291f4a17d3f6cefb6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:18:35 UTC) from {username='kudu', principal='kudu/127.15.75.190@KRBTEST.COM'} at 127.0.0.1:35822
I20260809 02:18:35.590054 5440 tablet_replica.cc:330] T c86bd422f6054e4291f4a17d3f6cefb6 P 352b7fb566bf458585a84dd49f8c07dc: stopping tablet replica
I20260809 02:18:35.590032 5749 tablet_service.cc:1514] Processing DeleteTablet for tablet c86bd422f6054e4291f4a17d3f6cefb6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:18:35 UTC) from {username='kudu', principal='kudu/127.15.75.190@KRBTEST.COM'} at 127.0.0.1:60710
I20260809 02:18:35.590154 5440 raft_consensus.cc:2229] T c86bd422f6054e4291f4a17d3f6cefb6 P 352b7fb566bf458585a84dd49f8c07dc [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:18:35.590435 5440 raft_consensus.cc:2258] T c86bd422f6054e4291f4a17d3f6cefb6 P 352b7fb566bf458585a84dd49f8c07dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:18:35.590528 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 5202
I20260809 02:18:35.605326 5895 tablet_replica.cc:330] T c86bd422f6054e4291f4a17d3f6cefb6 P d295396cdcae4b57acab477dfe1bd61d: stopping tablet replica
I20260809 02:18:35.605381 5898 tablet_replica.cc:330] T c86bd422f6054e4291f4a17d3f6cefb6 P 36d5cac71f2a4ef9bbfee682d6c3f712: stopping tablet replica
I20260809 02:18:35.605759 5898 raft_consensus.cc:2229] T c86bd422f6054e4291f4a17d3f6cefb6 P 36d5cac71f2a4ef9bbfee682d6c3f712 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:18:35.605760 5895 raft_consensus.cc:2229] T c86bd422f6054e4291f4a17d3f6cefb6 P d295396cdcae4b57acab477dfe1bd61d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:18:35.606135 5898 raft_consensus.cc:2258] T c86bd422f6054e4291f4a17d3f6cefb6 P 36d5cac71f2a4ef9bbfee682d6c3f712 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:18:35.606135 5895 raft_consensus.cc:2258] T c86bd422f6054e4291f4a17d3f6cefb6 P d295396cdcae4b57acab477dfe1bd61d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:18:35.607695 5895 ts_tablet_manager.cc:1905] T c86bd422f6054e4291f4a17d3f6cefb6 P d295396cdcae4b57acab477dfe1bd61d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:18:35.607785 5898 ts_tablet_manager.cc:1905] T c86bd422f6054e4291f4a17d3f6cefb6 P 36d5cac71f2a4ef9bbfee682d6c3f712: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:18:35.610085 5895 ts_tablet_manager.cc:1918] T c86bd422f6054e4291f4a17d3f6cefb6 P d295396cdcae4b57acab477dfe1bd61d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:18:35.610221 5895 log.cc:1196] T c86bd422f6054e4291f4a17d3f6cefb6 P d295396cdcae4b57acab477dfe1bd61d: Deleting WAL directory at /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-1/wal/wals/c86bd422f6054e4291f4a17d3f6cefb6
I20260809 02:18:35.610582 5895 ts_tablet_manager.cc:1939] T c86bd422f6054e4291f4a17d3f6cefb6 P d295396cdcae4b57acab477dfe1bd61d: Deleting consensus metadata
I20260809 02:18:35.611310 5898 ts_tablet_manager.cc:1918] T c86bd422f6054e4291f4a17d3f6cefb6 P 36d5cac71f2a4ef9bbfee682d6c3f712: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:18:35.611438 5898 log.cc:1196] T c86bd422f6054e4291f4a17d3f6cefb6 P 36d5cac71f2a4ef9bbfee682d6c3f712: Deleting WAL directory at /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-2/wal/wals/c86bd422f6054e4291f4a17d3f6cefb6
I20260809 02:18:35.611814 5898 ts_tablet_manager.cc:1939] T c86bd422f6054e4291f4a17d3f6cefb6 P 36d5cac71f2a4ef9bbfee682d6c3f712: Deleting consensus metadata
I20260809 02:18:35.611831 5128 catalog_manager.cc:4841] TS d295396cdcae4b57acab477dfe1bd61d (127.15.75.130:46695): tablet c86bd422f6054e4291f4a17d3f6cefb6 (table test-table [id=6b6cf70f0d484274b68aeac025d5fde2]) successfully deleted
I20260809 02:18:35.613078 5128 catalog_manager.cc:4841] TS 36d5cac71f2a4ef9bbfee682d6c3f712 (127.15.75.131:46171): tablet c86bd422f6054e4291f4a17d3f6cefb6 (table test-table [id=6b6cf70f0d484274b68aeac025d5fde2]) successfully deleted
W20260809 02:18:35.613487 5132 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.75.129:42815 (error 108)
I20260809 02:18:35.613866 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 5435
I20260809 02:18:35.620144 5864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:35.619817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.129:42815 (local address 127.0.0.1:40886)
0809 02:18:35.619974 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:35.620045 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.129:42815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260809 02:18:35.620313 5132 catalog_manager.cc:4584] TS 352b7fb566bf458585a84dd49f8c07dc (127.15.75.129:42815): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c86bd422f6054e4291f4a17d3f6cefb6: Network error: Client connection negotiation failed: client connection to 127.15.75.129:42815: connect: Connection refused (error 111)
I20260809 02:18:35.627030 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 5648
I20260809 02:18:35.636509 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 5105
I20260809 02:18:35.648275 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:35621
--webserver_interface=127.15.75.190
--webserver_port=45965
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab
--principal=kudu/127.15.75.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:42113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.75.190:35621
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:18:35.761714 5902 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:18:35.761953 5902 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:18:35.761994 5902 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:18:35.765923 5902 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:18:35.765974 5902 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:18:35.765993 5902 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:18:35.766010 5902 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:18:35.766026 5902 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:18:35.770969 5902 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:42113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.75.190:35621
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab
--principal=kudu/127.15.75.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:35621
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.75.190
--webserver_port=45965
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5902
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:18:35.771875 5902 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:18:35.772858 5902 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:18:35.780992 5909 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:35.781131 5902 server_base.cc:1034] running on GCE node
W20260809 02:18:35.781018 5914 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:35.781315 5910 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:35.781833 5902 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:18:35.782882 5902 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:18:35.784081 5902 hybrid_clock.cc:648] HybridClock initialized: now 1786241915784051 us; error 41 us; skew 500 ppm
Aug 09 02:18:35 dist-test-slave-s11t krb5kdc[4258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241915, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:18:35.787024 5902 init.cc:377] Logged in from keytab as kudu/127.15.75.190@KRBTEST.COM (short username kudu)
I20260809 02:18:35.788021 5902 webserver.cc:460] Webserver started at http://127.15.75.190:45965/ using document root <none> and password file <none>
I20260809 02:18:35.788580 5902 fs_manager.cc:359] Metadata directory not provided
I20260809 02:18:35.788676 5902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:18:35.792431 5902 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260809 02:18:35.794678 5925 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:35.795849 5902 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 02:18:35.796008 5902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/master-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/master-0/wal
uuid: "84a663e56c0248d1afacc5b1b6bf0298"
format_stamp: "Formatted at 2026-08-09 02:18:33 on dist-test-slave-s11t"
server_key: "04e8b42f6479cf489e38a750b41650d7"
server_key_iv: "8a24440a2521a44c67a8ed3ab63a5a71"
server_key_version: "encryptionkey@0"
I20260809 02:18:35.796542 5902 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:18:35.817132 5902 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:18:35.819772 5902 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:18:35.820003 5902 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:18:35.827481 5902 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.190:35621
I20260809 02:18:35.827769 6012 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.190:35621 every 8 connection(s)
I20260809 02:18:35.828589 5902 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
I20260809 02:18:35.832702 6014 sys_catalog.cc:263] Verifying existing consensus state
I20260809 02:18:35.834282 6014 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84a663e56c0248d1afacc5b1b6bf0298: Bootstrap starting.
I20260809 02:18:35.834738 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 5902
I20260809 02:18:35.835175 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.129:42815
--local_ip_for_outbound_sockets=127.15.75.129
--tserver_master_addrs=127.15.75.190:35621
--webserver_port=40793
--webserver_interface=127.15.75.129
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab
--principal=kudu/127.15.75.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:42113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 02:18:35.851575 6014 log.cc:824] T 00000000000000000000000000000000 P 84a663e56c0248d1afacc5b1b6bf0298: Log is configured to *not* fsync() on all Append() calls
I20260809 02:18:35.862890 6014 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84a663e56c0248d1afacc5b1b6bf0298: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 02:18:35.863265 6014 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84a663e56c0248d1afacc5b1b6bf0298: Bootstrap complete.
I20260809 02:18:35.867301 6014 raft_consensus.cc:348] T 00000000000000000000000000000000 P 84a663e56c0248d1afacc5b1b6bf0298 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84a663e56c0248d1afacc5b1b6bf0298" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 35621 } }
I20260809 02:18:35.867658 6014 raft_consensus.cc:729] T 00000000000000000000000000000000 P 84a663e56c0248d1afacc5b1b6bf0298 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84a663e56c0248d1afacc5b1b6bf0298, State: Initialized, Role: FOLLOWER
I20260809 02:18:35.868197 6014 consensus_queue.cc:260] T 00000000000000000000000000000000 P 84a663e56c0248d1afacc5b1b6bf0298 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84a663e56c0248d1afacc5b1b6bf0298" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 35621 } }
I20260809 02:18:35.868315 6014 raft_consensus.cc:388] T 00000000000000000000000000000000 P 84a663e56c0248d1afacc5b1b6bf0298 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:18:35.868402 6014 raft_consensus.cc:482] T 00000000000000000000000000000000 P 84a663e56c0248d1afacc5b1b6bf0298 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:18:35.868537 6014 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 84a663e56c0248d1afacc5b1b6bf0298 [term 2 FOLLOWER]: Advancing to term 3
I20260809 02:18:35.870409 6014 raft_consensus.cc:504] T 00000000000000000000000000000000 P 84a663e56c0248d1afacc5b1b6bf0298 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84a663e56c0248d1afacc5b1b6bf0298" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 35621 } }
I20260809 02:18:35.870738 6014 leader_election.cc:302] T 00000000000000000000000000000000 P 84a663e56c0248d1afacc5b1b6bf0298 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 84a663e56c0248d1afacc5b1b6bf0298; no voters:
I20260809 02:18:35.871043 6014 leader_election.cc:288] T 00000000000000000000000000000000 P 84a663e56c0248d1afacc5b1b6bf0298 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 02:18:35.871177 6019 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 84a663e56c0248d1afacc5b1b6bf0298 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 02:18:35.871438 6019 raft_consensus.cc:686] T 00000000000000000000000000000000 P 84a663e56c0248d1afacc5b1b6bf0298 [term 3 LEADER]: Becoming Leader. State: Replica: 84a663e56c0248d1afacc5b1b6bf0298, State: Running, Role: LEADER
I20260809 02:18:35.871752 6019 consensus_queue.cc:237] T 00000000000000000000000000000000 P 84a663e56c0248d1afacc5b1b6bf0298 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84a663e56c0248d1afacc5b1b6bf0298" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 35621 } }
I20260809 02:18:35.872241 6014 sys_catalog.cc:564] T 00000000000000000000000000000000 P 84a663e56c0248d1afacc5b1b6bf0298 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:18:35.873330 6021 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84a663e56c0248d1afacc5b1b6bf0298 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 84a663e56c0248d1afacc5b1b6bf0298. Latest consensus state: current_term: 3 leader_uuid: "84a663e56c0248d1afacc5b1b6bf0298" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84a663e56c0248d1afacc5b1b6bf0298" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 35621 } } }
I20260809 02:18:35.873458 6021 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84a663e56c0248d1afacc5b1b6bf0298 [sys.catalog]: This master's current role is: LEADER
I20260809 02:18:35.873668 6020 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84a663e56c0248d1afacc5b1b6bf0298 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "84a663e56c0248d1afacc5b1b6bf0298" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84a663e56c0248d1afacc5b1b6bf0298" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 35621 } } }
I20260809 02:18:35.873759 6020 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84a663e56c0248d1afacc5b1b6bf0298 [sys.catalog]: This master's current role is: LEADER
I20260809 02:18:35.877859 6042 catalog_manager.cc:1259] Loaded cluster ID: 68aea2d3fcfe4278a84f4fdaa2e4afae
I20260809 02:18:35.877918 6042 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 84a663e56c0248d1afacc5b1b6bf0298: loading cluster ID for follower catalog manager: success
W20260809 02:18:35.879063 6042 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 84a663e56c0248d1afacc5b1b6bf0298: acquiring CA information for follower catalog manager: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
W20260809 02:18:35.879138 6042 catalog_manager.cc:873] Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO: failed to prepare follower catalog manager, will retry
I20260809 02:18:35.879209 6026 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:18:35.881378 6026 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:18:35.881947 6026 catalog_manager.cc:1259] Loaded cluster ID: 68aea2d3fcfe4278a84f4fdaa2e4afae
I20260809 02:18:35.882009 6026 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260809 02:18:35.882649 6026 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fab5024d9ad google::LogMessage::Fail() at ??:0
@ 0x7fab5025088a google::LogMessage::SendToLog() at ??:0
@ 0x7fab5024d4de google::LogMessage::Flush() at ??:0
@ 0x7fab5024ea19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fab5ab8cfdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7fab5ab8dd2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fab5ab8a579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7fab5abce640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fab5249f3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fab5249fc83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7fab524a13f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fab52490512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fab53ad46db start_thread at ??:0
@ 0x7fab4ed4b61f clone at ??:0
W20260809 02:18:35.976624 6016 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:18:35.976933 6016 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:18:35.977000 6016 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:18:35.981020 6016 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:18:35.981067 6016 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:18:35.981145 6016 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.129
I20260809 02:18:35.988842 6016 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:42113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab
--principal=kudu/127.15.75.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.129:42815
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.75.129
--webserver_port=40793
--tserver_master_addrs=127.15.75.190:35621
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6016
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.129
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:18:35.990175 6016 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:18:35.991729 6016 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:18:36.003477 6083 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:36.003432 6076 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:36.003448 6077 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:36.004410 6016 server_base.cc:1034] running on GCE node
I20260809 02:18:36.004869 6016 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:18:36.005446 6016 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:18:36.006655 6016 hybrid_clock.cc:648] HybridClock initialized: now 1786241916006631 us; error 42 us; skew 500 ppm
Aug 09 02:18:36 dist-test-slave-s11t krb5kdc[4258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241916, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:18:36.009716 6016 init.cc:377] Logged in from keytab as kudu/127.15.75.129@KRBTEST.COM (short username kudu)
I20260809 02:18:36.010655 6016 webserver.cc:460] Webserver started at http://127.15.75.129:40793/ using document root <none> and password file <none>
I20260809 02:18:36.011220 6016 fs_manager.cc:359] Metadata directory not provided
I20260809 02:18:36.011286 6016 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:18:36.015249 6016 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 02:18:36.017657 6092 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:36.019012 6016 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 02:18:36.019150 6016 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-0/wal
uuid: "352b7fb566bf458585a84dd49f8c07dc"
format_stamp: "Formatted at 2026-08-09 02:18:33 on dist-test-slave-s11t"
server_key: "c40cb3aae85a366be328e883500d52f8"
server_key_iv: "6876e9e96b4ce7b076beac3486380a8b"
server_key_version: "encryptionkey@0"
I20260809 02:18:36.019695 6016 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:18:36.034212 6016 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:18:36.037254 6016 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:18:36.037443 6016 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:18:36.038003 6016 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:18:36.039074 6104 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 02:18:36.044859 6016 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 02:18:36.044946 6016 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260809 02:18:36.045022 6016 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 02:18:36.046667 6016 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 02:18:36.046737 6016 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260809 02:18:36.047034 6104 tablet_bootstrap.cc:492] T c86bd422f6054e4291f4a17d3f6cefb6 P 352b7fb566bf458585a84dd49f8c07dc: Bootstrap starting.
I20260809 02:18:36.057605 6016 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.129:42815
I20260809 02:18:36.057709 6267 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.129:42815 every 8 connection(s)
I20260809 02:18:36.058547 6016 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
I20260809 02:18:36.065296 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 6016
I20260809 02:18:36.065722 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.130:46695
--local_ip_for_outbound_sockets=127.15.75.130
--tserver_master_addrs=127.15.75.190:35621
--webserver_port=42913
--webserver_interface=127.15.75.130
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab
--principal=kudu/127.15.75.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:42113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 02:18:36.069430 6104 log.cc:824] T c86bd422f6054e4291f4a17d3f6cefb6 P 352b7fb566bf458585a84dd49f8c07dc: Log is configured to *not* fsync() on all Append() calls
I20260809 02:18:36.079540 6104 tablet_bootstrap.cc:492] T c86bd422f6054e4291f4a17d3f6cefb6 P 352b7fb566bf458585a84dd49f8c07dc: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 02:18:36.079969 6104 tablet_bootstrap.cc:492] T c86bd422f6054e4291f4a17d3f6cefb6 P 352b7fb566bf458585a84dd49f8c07dc: Bootstrap complete.
I20260809 02:18:36.081588 6104 ts_tablet_manager.cc:1397] T c86bd422f6054e4291f4a17d3f6cefb6 P 352b7fb566bf458585a84dd49f8c07dc: Time spent bootstrapping tablet: real 0.035s user 0.023s sys 0.011s
I20260809 02:18:36.084893 6104 raft_consensus.cc:348] T c86bd422f6054e4291f4a17d3f6cefb6 P 352b7fb566bf458585a84dd49f8c07dc [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d295396cdcae4b57acab477dfe1bd61d" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 46695 } } peers { permanent_uuid: "36d5cac71f2a4ef9bbfee682d6c3f712" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 46171 } } peers { permanent_uuid: "352b7fb566bf458585a84dd49f8c07dc" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 42815 } }
I20260809 02:18:36.085448 6104 raft_consensus.cc:729] T c86bd422f6054e4291f4a17d3f6cefb6 P 352b7fb566bf458585a84dd49f8c07dc [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 352b7fb566bf458585a84dd49f8c07dc, State: Initialized, Role: FOLLOWER
I20260809 02:18:36.085973 6104 consensus_queue.cc:260] T c86bd422f6054e4291f4a17d3f6cefb6 P 352b7fb566bf458585a84dd49f8c07dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d295396cdcae4b57acab477dfe1bd61d" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 46695 } } peers { permanent_uuid: "36d5cac71f2a4ef9bbfee682d6c3f712" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 46171 } } peers { permanent_uuid: "352b7fb566bf458585a84dd49f8c07dc" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 42815 } }
I20260809 02:18:36.087212 6104 ts_tablet_manager.cc:1428] T c86bd422f6054e4291f4a17d3f6cefb6 P 352b7fb566bf458585a84dd49f8c07dc: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
W20260809 02:18:36.161685 6271 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:18:36.060392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:35621 (local address 127.15.75.129:42111)
0809 02:18:36.060869 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:36.060919 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:36.061691 (+ 772us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:36.160941 (+ 99250us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:35621: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":138,"threads_started":1}
W20260809 02:18:36.162320 6268 heartbeater.cc:644] Failed to heartbeat to 127.15.75.190:35621 (0 consecutive failures): Network error: Failed to ping master at 127.15.75.190:35621: Client connection negotiation failed: client connection to 127.15.75.190:35621: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 02:18:36.163117 6271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:36.162878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:35621 (local address 127.15.75.129:54721)
0809 02:18:36.163004 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:36.163066 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:35621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260809 02:18:36.163769 6271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:36.163560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:35621 (local address 127.15.75.129:34849)
0809 02:18:36.163672 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:36.163722 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:35621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260809 02:18:36.164000 6268 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 02:18:36.185608 6273 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:18:36.185856 6273 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:18:36.185947 6273 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:18:36.189944 6273 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:18:36.190047 6273 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:18:36.190140 6273 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.130
I20260809 02:18:36.195256 6273 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:42113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab
--principal=kudu/127.15.75.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.130:46695
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.75.130
--webserver_port=42913
--tserver_master_addrs=127.15.75.190:35621
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6273
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.130
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:18:36.196209 6273 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:18:36.197202 6273 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:18:36.206388 6285 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:36.206400 6284 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:36.206375 6287 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:36.206709 6273 server_base.cc:1034] running on GCE node
I20260809 02:18:36.207093 6273 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:18:36.207716 6273 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:18:36.208896 6273 hybrid_clock.cc:648] HybridClock initialized: now 1786241916208876 us; error 30 us; skew 500 ppm
Aug 09 02:18:36 dist-test-slave-s11t krb5kdc[4258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241916, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:18:36.211540 6273 init.cc:377] Logged in from keytab as kudu/127.15.75.130@KRBTEST.COM (short username kudu)
I20260809 02:18:36.212383 6273 webserver.cc:460] Webserver started at http://127.15.75.130:42913/ using document root <none> and password file <none>
I20260809 02:18:36.212988 6273 fs_manager.cc:359] Metadata directory not provided
I20260809 02:18:36.213081 6273 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:18:36.216612 6273 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260809 02:18:36.218604 6296 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:36.219691 6273 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 02:18:36.219831 6273 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-1/wal
uuid: "d295396cdcae4b57acab477dfe1bd61d"
format_stamp: "Formatted at 2026-08-09 02:18:34 on dist-test-slave-s11t"
server_key: "f2f70264978910559b7a16b1d94caa58"
server_key_iv: "35a31c73d12962326cff4ca00a57858e"
server_key_version: "encryptionkey@0"
I20260809 02:18:36.220324 6273 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:18:36.230113 6273 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:18:36.232791 6273 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:18:36.232977 6273 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:18:36.233506 6273 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:18:36.234246 6273 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:18:36.234292 6273 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:36.234364 6273 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:18:36.234403 6273 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:36.243113 6273 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.130:46695
I20260809 02:18:36.243150 6456 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.130:46695 every 8 connection(s)
I20260809 02:18:36.244007 6273 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
I20260809 02:18:36.245260 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 6273
I20260809 02:18:36.245864 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.131:46171
--local_ip_for_outbound_sockets=127.15.75.131
--tserver_master_addrs=127.15.75.190:35621
--webserver_port=34579
--webserver_interface=127.15.75.131
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab
--principal=kudu/127.15.75.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:42113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 02:18:36.248997 6463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:36.246357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:35621 (local address 127.15.75.130:49741)
0809 02:18:36.247000 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:36.247205 (+ 205us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:35621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":187,"threads_started":1}
W20260809 02:18:36.249761 6460 heartbeater.cc:644] Failed to heartbeat to 127.15.75.190:35621 (0 consecutive failures): Network error: Failed to ping master at 127.15.75.190:35621: Client connection negotiation failed: client connection to 127.15.75.190:35621: connect: Connection refused (error 111)
I20260809 02:18:36.250437 6463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:36.250158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:35621 (local address 127.15.75.130:60945)
0809 02:18:36.250318 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:36.250369 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:35621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"mutex_wait_us":77}
I20260809 02:18:36.251211 6463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:36.250939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:35621 (local address 127.15.75.130:35879)
0809 02:18:36.251073 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:36.251141 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:35621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260809 02:18:36.251386 6460 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 02:18:36.358644 6464 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:18:36.358904 6464 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:18:36.358974 6464 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:18:36.362897 6464 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:18:36.362972 6464 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:18:36.363052 6464 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.131
I20260809 02:18:36.367982 6464 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:42113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab
--principal=kudu/127.15.75.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.131:46171
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.75.131
--webserver_port=34579
--tserver_master_addrs=127.15.75.190:35621
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6464
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.131
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:18:36.368963 6464 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:18:36.369889 6464 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:18:36.379032 6475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:36.379024 6470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:36.379181 6464 server_base.cc:1034] running on GCE node
W20260809 02:18:36.379024 6471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:36.379689 6464 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:18:36.380267 6464 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:18:36.381426 6464 hybrid_clock.cc:648] HybridClock initialized: now 1786241916381395 us; error 43 us; skew 500 ppm
Aug 09 02:18:36 dist-test-slave-s11t krb5kdc[4258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241916, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:18:36.384130 6464 init.cc:377] Logged in from keytab as kudu/127.15.75.131@KRBTEST.COM (short username kudu)
I20260809 02:18:36.384956 6464 webserver.cc:460] Webserver started at http://127.15.75.131:34579/ using document root <none> and password file <none>
I20260809 02:18:36.385439 6464 fs_manager.cc:359] Metadata directory not provided
I20260809 02:18:36.385494 6464 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:18:36.388906 6464 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260809 02:18:36.390904 6487 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:36.392066 6464 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 02:18:36.392159 6464 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-2/wal
uuid: "36d5cac71f2a4ef9bbfee682d6c3f712"
format_stamp: "Formatted at 2026-08-09 02:18:34 on dist-test-slave-s11t"
server_key: "417270fd0d930863d79dfbe3d92c882d"
server_key_iv: "88efa4f9f7d188aef74706e9b6e19694"
server_key_version: "encryptionkey@0"
I20260809 02:18:36.392632 6464 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:18:36.405941 6464 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:18:36.409029 6464 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:18:36.409169 6464 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:18:36.409650 6464 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:18:36.410368 6464 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:18:36.410421 6464 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:36.410459 6464 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:18:36.410478 6464 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:36.419425 6464 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.131:46171
I20260809 02:18:36.419445 6668 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.131:46171 every 8 connection(s)
I20260809 02:18:36.420351 6464 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
I20260809 02:18:36.424089 6673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:36.422348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:35621 (local address 127.15.75.131:40449)
0809 02:18:36.422819 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:36.423354 (+ 535us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:35621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":158,"threads_started":1}
W20260809 02:18:36.424664 6670 heartbeater.cc:644] Failed to heartbeat to 127.15.75.190:35621 (0 consecutive failures): Network error: Failed to ping master at 127.15.75.190:35621: Client connection negotiation failed: client connection to 127.15.75.190:35621: connect: Connection refused (error 111)
I20260809 02:18:36.425266 6673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:36.425073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:35621 (local address 127.15.75.131:37879)
0809 02:18:36.425172 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:36.425216 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:35621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260809 02:18:36.425523 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 6464
I20260809 02:18:36.425951 6673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:36.425722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:35621 (local address 127.15.75.131:35433)
0809 02:18:36.425835 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:36.425892 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:35621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260809 02:18:36.426170 6670 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 02:18:37.164939 6677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:37.164491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:35621 (local address 127.15.75.129:44635)
0809 02:18:37.164772 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:37.164823 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:35621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":72,"threads_started":1}
I20260809 02:18:37.252240 6678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:37.251877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:35621 (local address 127.15.75.130:60277)
0809 02:18:37.252095 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:37.252143 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:35621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":68,"threads_started":1}
I20260809 02:18:37.361897 6679 raft_consensus.cc:482] T c86bd422f6054e4291f4a17d3f6cefb6 P 352b7fb566bf458585a84dd49f8c07dc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:18:37.362141 6679 raft_consensus.cc:504] T c86bd422f6054e4291f4a17d3f6cefb6 P 352b7fb566bf458585a84dd49f8c07dc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d295396cdcae4b57acab477dfe1bd61d" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 46695 } } peers { permanent_uuid: "36d5cac71f2a4ef9bbfee682d6c3f712" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 46171 } } peers { permanent_uuid: "352b7fb566bf458585a84dd49f8c07dc" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 42815 } }
I20260809 02:18:37.363284 6679 leader_election.cc:288] T c86bd422f6054e4291f4a17d3f6cefb6 P 352b7fb566bf458585a84dd49f8c07dc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d295396cdcae4b57acab477dfe1bd61d (127.15.75.130:46695), 36d5cac71f2a4ef9bbfee682d6c3f712 (127.15.75.131:46171)
Aug 09 02:18:37 dist-test-slave-s11t krb5kdc[4258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241916, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.129@KRBTEST.COM for kudu/127.15.75.131@KRBTEST.COM
Aug 09 02:18:37 dist-test-slave-s11t krb5kdc[4258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241916, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.129@KRBTEST.COM for kudu/127.15.75.130@KRBTEST.COM
I20260809 02:18:37.382000 6677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:37.363458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.130:46695 (local address 127.15.75.129:46189)
0809 02:18:37.363591 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:37.363604 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:37.363707 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:37.365625 (+ 1918us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:37.365631 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:37.366174 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:18:37.366666 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:37.366685 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:37.368042 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:37.368048 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:37.369080 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:37.369090 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:37.369791 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:37.369795 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:37.370020 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:37.375469 (+ 5449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:18:37.375486 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:18:37.377558 (+ 2072us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:18:37.379673 (+ 2115us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:18:37.379679 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:18:37.379682 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:18:37.379968 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:18:37.380323 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:18:37.380327 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:18:37.380329 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:18:37.380415 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:18:37.381040 (+ 625us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:18:37.381049 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:18:37.381285 (+ 236us) client_negotiation.cc:769] Sending connection context
0809 02:18:37.381463 (+ 178us) client_negotiation.cc:240] Negotiation successful
0809 02:18:37.381815 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53,"mutex_wait_us":190,"spinlock_wait_cycles":38272}
I20260809 02:18:37.382012 6680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:37.363635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.129:46189 (local address 127.15.75.130:46695)
0809 02:18:37.364129 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:37.364156 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:37.364206 (+ 50us) server_negotiation.cc:407] Connection header received
0809 02:18:37.364923 (+ 717us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:37.364929 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:37.365260 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:37.365574 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:37.366858 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:37.367899 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:37.369202 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:37.369658 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:37.369830 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:37.377776 (+ 7946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:37.377780 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:37.377789 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:18:37.377816 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:37.379528 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:37.380087 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:37.380095 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:37.380104 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:37.380210 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:37.380511 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:37.380515 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:37.380517 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:37.380931 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:37.381070 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:37.381581 (+ 511us) server_negotiation.cc:299] Negotiation successful
0809 02:18:37.381833 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":143,"threads_started":1}
I20260809 02:18:37.382036 6683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:37.363936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.129:33967 (local address 127.15.75.131:46171)
0809 02:18:37.364271 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:37.364281 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:37.364314 (+ 33us) server_negotiation.cc:407] Connection header received
0809 02:18:37.364986 (+ 672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:37.364993 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:37.365285 (+ 292us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:37.365574 (+ 289us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:37.366868 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:37.367921 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:37.369208 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:37.369658 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:37.369829 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:37.377454 (+ 7625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:37.377464 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:37.377487 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:18:37.377516 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:37.379406 (+ 1890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:37.379996 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:37.380002 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:37.380007 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:37.380128 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:37.380464 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:37.380468 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:37.380470 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:37.380926 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:37.381066 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:37.381578 (+ 512us) server_negotiation.cc:299] Negotiation successful
0809 02:18:37.381849 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":95,"threads_started":1}
I20260809 02:18:37.382061 6682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:37.363729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.131:46171 (local address 127.15.75.129:33967)
0809 02:18:37.364017 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:37.364029 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:37.364151 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:37.365626 (+ 1475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:37.365632 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:37.366175 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:18:37.366665 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:37.366701 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:37.368049 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:37.368054 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:37.369085 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:37.369096 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:37.369781 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:37.369785 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:37.370022 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:37.375480 (+ 5458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:18:37.375492 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:18:37.377195 (+ 1703us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:18:37.379530 (+ 2335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:18:37.379536 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:18:37.379548 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:18:37.379872 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:18:37.380227 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:18:37.380231 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:18:37.380233 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:18:37.380367 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:18:37.381038 (+ 671us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:18:37.381045 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:18:37.381278 (+ 233us) client_negotiation.cc:769] Sending connection context
0809 02:18:37.381464 (+ 186us) client_negotiation.cc:240] Negotiation successful
0809 02:18:37.381904 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"mutex_wait_us":299,"spinlock_wait_cycles":16768,"thread_start_us":77,"threads_started":1}
W20260809 02:18:37.385011 6096 leader_election.cc:341] T c86bd422f6054e4291f4a17d3f6cefb6 P 352b7fb566bf458585a84dd49f8c07dc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 36d5cac71f2a4ef9bbfee682d6c3f712 (127.15.75.131:46171): Not found: Tablet not found: c86bd422f6054e4291f4a17d3f6cefb6
W20260809 02:18:37.385676 6096 leader_election.cc:341] T c86bd422f6054e4291f4a17d3f6cefb6 P 352b7fb566bf458585a84dd49f8c07dc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d295396cdcae4b57acab477dfe1bd61d (127.15.75.130:46695): Not found: Tablet not found: c86bd422f6054e4291f4a17d3f6cefb6
I20260809 02:18:37.385762 6096 leader_election.cc:302] T c86bd422f6054e4291f4a17d3f6cefb6 P 352b7fb566bf458585a84dd49f8c07dc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 352b7fb566bf458585a84dd49f8c07dc; no voters: 36d5cac71f2a4ef9bbfee682d6c3f712, d295396cdcae4b57acab477dfe1bd61d
I20260809 02:18:37.385990 6679 raft_consensus.cc:2726] T c86bd422f6054e4291f4a17d3f6cefb6 P 352b7fb566bf458585a84dd49f8c07dc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 02:18:37.427007 6684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:37.426643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:35621 (local address 127.15.75.131:37781)
0809 02:18:37.426875 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:37.426923 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:35621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":83,"threads_started":1}
I20260809 02:18:38.166103 6685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:38.165668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:35621 (local address 127.15.75.129:40771)
0809 02:18:38.165948 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:38.166005 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:35621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":86,"threads_started":1}
I20260809 02:18:38.253270 6686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:38.252922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:35621 (local address 127.15.75.130:52699)
0809 02:18:38.253140 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:38.253190 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:35621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":69,"threads_started":1}
I20260809 02:18:38.428006 6688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:38.427631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:35621 (local address 127.15.75.131:53251)
0809 02:18:38.427852 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:38.427912 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:35621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":73,"threads_started":1}
I20260809 02:18:39.167439 6689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:39.167050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:35621 (local address 127.15.75.129:35259)
0809 02:18:39.167290 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:39.167343 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:35621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":75,"threads_started":1}
I20260809 02:18:39.255649 6691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:39.255244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:35621 (local address 127.15.75.130:60229)
0809 02:18:39.255468 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:39.255528 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:35621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":85,"threads_started":1}
I20260809 02:18:39.429316 6692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:39.428902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:35621 (local address 127.15.75.131:35529)
0809 02:18:39.429168 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:39.429223 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:35621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":76,"threads_started":1}
I20260809 02:18:39.476999 6694 raft_consensus.cc:482] T c86bd422f6054e4291f4a17d3f6cefb6 P 352b7fb566bf458585a84dd49f8c07dc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:18:39.477134 6694 raft_consensus.cc:504] T c86bd422f6054e4291f4a17d3f6cefb6 P 352b7fb566bf458585a84dd49f8c07dc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d295396cdcae4b57acab477dfe1bd61d" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 46695 } } peers { permanent_uuid: "36d5cac71f2a4ef9bbfee682d6c3f712" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 46171 } } peers { permanent_uuid: "352b7fb566bf458585a84dd49f8c07dc" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 42815 } }
I20260809 02:18:39.477555 6694 leader_election.cc:288] T c86bd422f6054e4291f4a17d3f6cefb6 P 352b7fb566bf458585a84dd49f8c07dc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d295396cdcae4b57acab477dfe1bd61d (127.15.75.130:46695), 36d5cac71f2a4ef9bbfee682d6c3f712 (127.15.75.131:46171)
W20260809 02:18:39.478503 6096 leader_election.cc:341] T c86bd422f6054e4291f4a17d3f6cefb6 P 352b7fb566bf458585a84dd49f8c07dc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d295396cdcae4b57acab477dfe1bd61d (127.15.75.130:46695): Not found: Tablet not found: c86bd422f6054e4291f4a17d3f6cefb6
W20260809 02:18:39.478649 6096 leader_election.cc:341] T c86bd422f6054e4291f4a17d3f6cefb6 P 352b7fb566bf458585a84dd49f8c07dc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 36d5cac71f2a4ef9bbfee682d6c3f712 (127.15.75.131:46171): Not found: Tablet not found: c86bd422f6054e4291f4a17d3f6cefb6
I20260809 02:18:39.478782 6096 leader_election.cc:302] T c86bd422f6054e4291f4a17d3f6cefb6 P 352b7fb566bf458585a84dd49f8c07dc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 352b7fb566bf458585a84dd49f8c07dc; no voters: 36d5cac71f2a4ef9bbfee682d6c3f712, d295396cdcae4b57acab477dfe1bd61d
I20260809 02:18:39.478981 6694 raft_consensus.cc:2726] T c86bd422f6054e4291f4a17d3f6cefb6 P 352b7fb566bf458585a84dd49f8c07dc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 02:18:40.168605 6697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:40.168191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:35621 (local address 127.15.75.129:35197)
0809 02:18:40.168449 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:40.168509 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:35621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":78,"threads_started":1}
I20260809 02:18:40.256846 6698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:40.256412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:35621 (local address 127.15.75.130:47699)
0809 02:18:40.256643 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:40.256745 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:35621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":71,"threads_started":1}
I20260809 02:18:40.430467 6699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:40.430054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:35621 (local address 127.15.75.131:55399)
0809 02:18:40.430332 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:40.430381 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:35621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":99,"threads_started":1}
I20260809 02:18:41.170040 6700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:41.169562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:35621 (local address 127.15.75.129:44783)
0809 02:18:41.169888 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:41.169944 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:35621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":76,"threads_started":1}
I20260809 02:18:41.258028 6701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:41.257550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:35621 (local address 127.15.75.130:51643)
0809 02:18:41.257834 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:41.257921 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:35621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":104,"threads_started":1}
I20260809 02:18:41.431541 6702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:41.431130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:35621 (local address 127.15.75.131:49205)
0809 02:18:41.431377 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:41.431453 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:35621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":72,"threads_started":1}
I20260809 02:18:42.171380 6703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:42.170922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:35621 (local address 127.15.75.129:48567)
0809 02:18:42.171219 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:42.171280 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:35621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":78,"threads_started":1}
I20260809 02:18:42.259136 6704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:42.258739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:35621 (local address 127.15.75.130:59231)
0809 02:18:42.258993 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:42.259044 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:35621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":87,"threads_started":1}
I20260809 02:18:42.432665 6707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:42.432264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:35621 (local address 127.15.75.131:53225)
0809 02:18:42.432521 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:42.432577 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:35621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":74,"threads_started":1}
I20260809 02:18:42.518381 6708 raft_consensus.cc:482] T c86bd422f6054e4291f4a17d3f6cefb6 P 352b7fb566bf458585a84dd49f8c07dc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:18:42.518546 6708 raft_consensus.cc:504] T c86bd422f6054e4291f4a17d3f6cefb6 P 352b7fb566bf458585a84dd49f8c07dc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d295396cdcae4b57acab477dfe1bd61d" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 46695 } } peers { permanent_uuid: "36d5cac71f2a4ef9bbfee682d6c3f712" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 46171 } } peers { permanent_uuid: "352b7fb566bf458585a84dd49f8c07dc" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 42815 } }
I20260809 02:18:42.518955 6708 leader_election.cc:288] T c86bd422f6054e4291f4a17d3f6cefb6 P 352b7fb566bf458585a84dd49f8c07dc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d295396cdcae4b57acab477dfe1bd61d (127.15.75.130:46695), 36d5cac71f2a4ef9bbfee682d6c3f712 (127.15.75.131:46171)
W20260809 02:18:42.519879 6096 leader_election.cc:341] T c86bd422f6054e4291f4a17d3f6cefb6 P 352b7fb566bf458585a84dd49f8c07dc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 36d5cac71f2a4ef9bbfee682d6c3f712 (127.15.75.131:46171): Not found: Tablet not found: c86bd422f6054e4291f4a17d3f6cefb6
W20260809 02:18:42.520033 6096 leader_election.cc:341] T c86bd422f6054e4291f4a17d3f6cefb6 P 352b7fb566bf458585a84dd49f8c07dc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d295396cdcae4b57acab477dfe1bd61d (127.15.75.130:46695): Not found: Tablet not found: c86bd422f6054e4291f4a17d3f6cefb6
I20260809 02:18:42.520107 6096 leader_election.cc:302] T c86bd422f6054e4291f4a17d3f6cefb6 P 352b7fb566bf458585a84dd49f8c07dc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 352b7fb566bf458585a84dd49f8c07dc; no voters: 36d5cac71f2a4ef9bbfee682d6c3f712, d295396cdcae4b57acab477dfe1bd61d
I20260809 02:18:42.520284 6708 raft_consensus.cc:2726] T c86bd422f6054e4291f4a17d3f6cefb6 P 352b7fb566bf458585a84dd49f8c07dc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 02:18:43.172735 6709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:43.172264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:35621 (local address 127.15.75.129:38813)
0809 02:18:43.172530 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:43.172584 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:35621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":101,"threads_started":1}
I20260809 02:18:43.260262 6710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:43.259831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:35621 (local address 127.15.75.130:45269)
0809 02:18:43.260121 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:43.260174 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:35621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":67,"threads_started":1}
I20260809 02:18:43.433990 6711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:43.433581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:35621 (local address 127.15.75.131:47855)
0809 02:18:43.433851 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:43.433903 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:35621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":69,"threads_started":1}
I20260809 02:18:44.173936 6712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:44.173513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:35621 (local address 127.15.75.129:60447)
0809 02:18:44.173779 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:44.173838 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:35621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":75,"threads_started":1}
I20260809 02:18:44.261461 6713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:44.261072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:35621 (local address 127.15.75.130:38423)
0809 02:18:44.261314 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:44.261368 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:35621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":71,"threads_started":1}
I20260809 02:18:44.435182 6716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:44.434735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:35621 (local address 127.15.75.131:57195)
0809 02:18:44.435042 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:44.435093 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:35621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":103,"threads_started":1}
I20260809 02:18:45.175151 6717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:45.174748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:35621 (local address 127.15.75.129:41373)
0809 02:18:45.175001 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:45.175053 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:35621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":75,"threads_started":1}
I20260809 02:18:45.262562 6718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:45.262169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:35621 (local address 127.15.75.130:35329)
0809 02:18:45.262423 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:45.262477 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:35621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":76,"threads_started":1}
I20260809 02:18:45.436445 6719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:45.436004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:35621 (local address 127.15.75.131:57517)
0809 02:18:45.436293 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:45.436349 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:35621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":100,"threads_started":1}
I20260809 02:18:46.176357 6720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:46.175978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:35621 (local address 127.15.75.129:41541)
0809 02:18:46.176211 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:46.176264 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:35621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":74,"threads_started":1}
I20260809 02:18:46.263622 6721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:46.263256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:35621 (local address 127.15.75.130:55221)
0809 02:18:46.263480 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:46.263533 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:35621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":72,"threads_started":1}
I20260809 02:18:46.437590 6723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:46.437207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:35621 (local address 127.15.75.131:53207)
0809 02:18:46.437449 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:46.437502 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:35621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":73,"threads_started":1}
I20260809 02:18:47.177515 6727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:47.177102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:35621 (local address 127.15.75.129:59919)
0809 02:18:47.177357 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:47.177419 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:35621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":90,"threads_started":1}
I20260809 02:18:47.264729 6728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:47.264268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:35621 (local address 127.15.75.130:42357)
0809 02:18:47.264498 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:47.264551 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:35621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":72,"threads_started":1}
I20260809 02:18:47.438661 6729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:47.438307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:35621 (local address 127.15.75.131:60671)
0809 02:18:47.438530 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:47.438579 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:35621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":74,"threads_started":1}
I20260809 02:18:47.488981 6731 raft_consensus.cc:482] T c86bd422f6054e4291f4a17d3f6cefb6 P 352b7fb566bf458585a84dd49f8c07dc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:18:47.489113 6731 raft_consensus.cc:504] T c86bd422f6054e4291f4a17d3f6cefb6 P 352b7fb566bf458585a84dd49f8c07dc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d295396cdcae4b57acab477dfe1bd61d" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 46695 } } peers { permanent_uuid: "36d5cac71f2a4ef9bbfee682d6c3f712" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 46171 } } peers { permanent_uuid: "352b7fb566bf458585a84dd49f8c07dc" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 42815 } }
I20260809 02:18:47.489611 6731 leader_election.cc:288] T c86bd422f6054e4291f4a17d3f6cefb6 P 352b7fb566bf458585a84dd49f8c07dc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d295396cdcae4b57acab477dfe1bd61d (127.15.75.130:46695), 36d5cac71f2a4ef9bbfee682d6c3f712 (127.15.75.131:46171)
W20260809 02:18:47.490403 6096 leader_election.cc:341] T c86bd422f6054e4291f4a17d3f6cefb6 P 352b7fb566bf458585a84dd49f8c07dc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 36d5cac71f2a4ef9bbfee682d6c3f712 (127.15.75.131:46171): Not found: Tablet not found: c86bd422f6054e4291f4a17d3f6cefb6
W20260809 02:18:47.490567 6096 leader_election.cc:341] T c86bd422f6054e4291f4a17d3f6cefb6 P 352b7fb566bf458585a84dd49f8c07dc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d295396cdcae4b57acab477dfe1bd61d (127.15.75.130:46695): Not found: Tablet not found: c86bd422f6054e4291f4a17d3f6cefb6
I20260809 02:18:47.490635 6096 leader_election.cc:302] T c86bd422f6054e4291f4a17d3f6cefb6 P 352b7fb566bf458585a84dd49f8c07dc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 352b7fb566bf458585a84dd49f8c07dc; no voters: 36d5cac71f2a4ef9bbfee682d6c3f712, d295396cdcae4b57acab477dfe1bd61d
I20260809 02:18:47.490773 6731 raft_consensus.cc:2726] T c86bd422f6054e4291f4a17d3f6cefb6 P 352b7fb566bf458585a84dd49f8c07dc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 02:18:48.178658 6732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:48.178246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:35621 (local address 127.15.75.129:35141)
0809 02:18:48.178509 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:48.178563 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:35621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":96,"threads_started":1}
I20260809 02:18:48.265892 6733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:48.265446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:35621 (local address 127.15.75.130:53543)
0809 02:18:48.265742 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:48.265804 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:35621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":111,"threads_started":1}
I20260809 02:18:48.439760 6734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:48.439384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:35621 (local address 127.15.75.131:42983)
0809 02:18:48.439614 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:48.439665 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:35621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":71,"threads_started":1}
I20260809 02:18:49.179901 6736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:49.179394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:35621 (local address 127.15.75.129:47653)
0809 02:18:49.179735 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:49.179805 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:35621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":113,"threads_started":1}
I20260809 02:18:49.267040 6737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:49.266634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:35621 (local address 127.15.75.130:60727)
0809 02:18:49.266882 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:49.266943 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:35621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":73,"threads_started":1}
I20260809 02:18:49.441000 6739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:49.440489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:35621 (local address 127.15.75.131:40941)
0809 02:18:49.440831 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:49.440897 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:35621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":89,"threads_started":1}
I20260809 02:18:50.181139 6741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:50.180623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:35621 (local address 127.15.75.129:56479)
0809 02:18:50.180987 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:50.181044 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:35621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":143,"threads_started":1}
I20260809 02:18:50.268263 6742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:50.267829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:35621 (local address 127.15.75.130:50979)
0809 02:18:50.268113 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:50.268168 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:35621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":99,"threads_started":1}
I20260809 02:18:50.442153 6743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:50.441758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:35621 (local address 127.15.75.131:36947)
0809 02:18:50.442009 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:50.442061 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:35621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":92,"threads_started":1}
I20260809 02:18:51.115882 6745 raft_consensus.cc:482] T c86bd422f6054e4291f4a17d3f6cefb6 P 352b7fb566bf458585a84dd49f8c07dc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:18:51.116017 6745 raft_consensus.cc:504] T c86bd422f6054e4291f4a17d3f6cefb6 P 352b7fb566bf458585a84dd49f8c07dc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d295396cdcae4b57acab477dfe1bd61d" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 46695 } } peers { permanent_uuid: "36d5cac71f2a4ef9bbfee682d6c3f712" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 46171 } } peers { permanent_uuid: "352b7fb566bf458585a84dd49f8c07dc" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 42815 } }
I20260809 02:18:51.116479 6745 leader_election.cc:288] T c86bd422f6054e4291f4a17d3f6cefb6 P 352b7fb566bf458585a84dd49f8c07dc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d295396cdcae4b57acab477dfe1bd61d (127.15.75.130:46695), 36d5cac71f2a4ef9bbfee682d6c3f712 (127.15.75.131:46171)
W20260809 02:18:51.117300 6096 leader_election.cc:341] T c86bd422f6054e4291f4a17d3f6cefb6 P 352b7fb566bf458585a84dd49f8c07dc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d295396cdcae4b57acab477dfe1bd61d (127.15.75.130:46695): Not found: Tablet not found: c86bd422f6054e4291f4a17d3f6cefb6
W20260809 02:18:51.117499 6096 leader_election.cc:341] T c86bd422f6054e4291f4a17d3f6cefb6 P 352b7fb566bf458585a84dd49f8c07dc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 36d5cac71f2a4ef9bbfee682d6c3f712 (127.15.75.131:46171): Not found: Tablet not found: c86bd422f6054e4291f4a17d3f6cefb6
I20260809 02:18:51.117555 6096 leader_election.cc:302] T c86bd422f6054e4291f4a17d3f6cefb6 P 352b7fb566bf458585a84dd49f8c07dc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 352b7fb566bf458585a84dd49f8c07dc; no voters: 36d5cac71f2a4ef9bbfee682d6c3f712, d295396cdcae4b57acab477dfe1bd61d
I20260809 02:18:51.117692 6745 raft_consensus.cc:2726] T c86bd422f6054e4291f4a17d3f6cefb6 P 352b7fb566bf458585a84dd49f8c07dc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 02:18:51.182468 6746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:51.182083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:35621 (local address 127.15.75.129:54109)
0809 02:18:51.182318 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:51.182370 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:35621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":78,"threads_started":1}
I20260809 02:18:51.269636 6748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:51.269223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:35621 (local address 127.15.75.130:56313)
0809 02:18:51.269481 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:51.269542 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:35621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":94,"threads_started":1}
I20260809 02:18:51.432085 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 6016
I20260809 02:18:51.443634 6749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:51.443138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:35621 (local address 127.15.75.131:55051)
0809 02:18:51.443421 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:51.443496 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:35621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":69,"threads_started":1}
I20260809 02:18:51.450712 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 6273
I20260809 02:18:51.462508 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 6464
2026-08-09T02:18:51Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19363 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:18:51 dist-test-slave-s11t krb5kdc[6753](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:18:51 dist-test-slave-s11t krb5kdc[6753](info): set up 2 sockets
Aug 09 02:18:51 dist-test-slave-s11t krb5kdc[6753](info): commencing operation
krb5kdc: starting...
W20260809 02:18:52.697630 15662 mini_kdc.cc:121] Time spent starting KDC: real 1.206s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Aug 09 02:18:52 dist-test-slave-s11t krb5kdc[6753](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241932, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:18:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:18:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.75.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab.
Entry for principal kudu/127.15.75.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab.
I20260809 02:18:52.819638 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:36755
--webserver_interface=127.15.75.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab
--principal=kudu/127.15.75.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:45655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.75.190:36755
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:18:52.930111 6775 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:18:52.930404 6775 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:18:52.930478 6775 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:18:52.934392 6775 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 02:18:52.934466 6775 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 02:18:52.934494 6775 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 02:18:52.934525 6775 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:18:52.934567 6775 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:18:52.934612 6775 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:18:52.934648 6775 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:18:52.934679 6775 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:18:52.939706 6775 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:45655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.15.75.190:36755
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab
--principal=kudu/127.15.75.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:36755
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.75.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6775
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:18:52.940733 6775 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:18:52.941715 6775 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:18:52.950480 6786 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:52.950464 6784 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:52.950519 6775 server_base.cc:1034] running on GCE node
W20260809 02:18:52.950464 6788 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:52.951256 6775 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:18:52.952265 6775 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:18:52.953464 6775 hybrid_clock.cc:648] HybridClock initialized: now 1786241932953446 us; error 31 us; skew 500 ppm
Aug 09 02:18:52 dist-test-slave-s11t krb5kdc[6753](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241932, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:18:52.956118 6775 init.cc:377] Logged in from keytab as kudu/127.15.75.190@KRBTEST.COM (short username kudu)
I20260809 02:18:52.957058 6775 webserver.cc:460] Webserver started at http://127.15.75.190:36907/ using document root <none> and password file <none>
I20260809 02:18:52.957567 6775 fs_manager.cc:359] Metadata directory not provided
I20260809 02:18:52.957624 6775 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:18:52.957783 6775 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:18:52.959685 6775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/master-0/data/instance:
uuid: "a708a9aa21ba4915b9ae3f1ff4de527c"
format_stamp: "Formatted at 2026-08-09 02:18:52 on dist-test-slave-s11t"
server_key: "5d58d775c4e31ddf3fd36fd7b0e4d8ad"
server_key_iv: "193ea38e7521fe51bb2e9302c4109c7f"
server_key_version: "encryptionkey@0"
I20260809 02:18:52.960176 6775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/master-0/wal/instance:
uuid: "a708a9aa21ba4915b9ae3f1ff4de527c"
format_stamp: "Formatted at 2026-08-09 02:18:52 on dist-test-slave-s11t"
server_key: "5d58d775c4e31ddf3fd36fd7b0e4d8ad"
server_key_iv: "193ea38e7521fe51bb2e9302c4109c7f"
server_key_version: "encryptionkey@0"
I20260809 02:18:52.963677 6775 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260809 02:18:52.965973 6800 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:52.967165 6775 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 02:18:52.967263 6775 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/master-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/master-0/wal
uuid: "a708a9aa21ba4915b9ae3f1ff4de527c"
format_stamp: "Formatted at 2026-08-09 02:18:52 on dist-test-slave-s11t"
server_key: "5d58d775c4e31ddf3fd36fd7b0e4d8ad"
server_key_iv: "193ea38e7521fe51bb2e9302c4109c7f"
server_key_version: "encryptionkey@0"
I20260809 02:18:52.967357 6775 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:18:52.988994 6775 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:18:52.991969 6775 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:18:52.992128 6775 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:18:52.999384 6775 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.190:36755
I20260809 02:18:52.999670 6884 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.190:36755 every 8 connection(s)
I20260809 02:18:53.000367 6775 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
I20260809 02:18:53.003197 6887 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:18:53.005834 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 6775
I20260809 02:18:53.005981 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/master-0/wal/instance
I20260809 02:18:53.006311 15662 external_mini_cluster.cc:1428] Setting key 7772fd5feec937f515f945fd9acef287
I20260809 02:18:53.009445 6887 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a708a9aa21ba4915b9ae3f1ff4de527c: Bootstrap starting.
I20260809 02:18:53.011953 6887 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a708a9aa21ba4915b9ae3f1ff4de527c: Neither blocks nor log segments found. Creating new log.
I20260809 02:18:53.012852 6887 log.cc:824] T 00000000000000000000000000000000 P a708a9aa21ba4915b9ae3f1ff4de527c: Log is configured to *not* fsync() on all Append() calls
I20260809 02:18:53.015179 6887 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a708a9aa21ba4915b9ae3f1ff4de527c: No bootstrap required, opened a new log
Aug 09 02:18:53 dist-test-slave-s11t krb5kdc[6753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241932, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:18:53.018666 6887 raft_consensus.cc:348] T 00000000000000000000000000000000 P a708a9aa21ba4915b9ae3f1ff4de527c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a708a9aa21ba4915b9ae3f1ff4de527c" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 36755 } }
I20260809 02:18:53.018903 6887 raft_consensus.cc:374] T 00000000000000000000000000000000 P a708a9aa21ba4915b9ae3f1ff4de527c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:18:53.018983 6887 raft_consensus.cc:729] T 00000000000000000000000000000000 P a708a9aa21ba4915b9ae3f1ff4de527c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a708a9aa21ba4915b9ae3f1ff4de527c, State: Initialized, Role: FOLLOWER
I20260809 02:18:53.019418 6887 consensus_queue.cc:260] T 00000000000000000000000000000000 P a708a9aa21ba4915b9ae3f1ff4de527c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a708a9aa21ba4915b9ae3f1ff4de527c" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 36755 } }
I20260809 02:18:53.019569 6887 raft_consensus.cc:388] T 00000000000000000000000000000000 P a708a9aa21ba4915b9ae3f1ff4de527c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:18:53.019676 6887 raft_consensus.cc:482] T 00000000000000000000000000000000 P a708a9aa21ba4915b9ae3f1ff4de527c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:18:53.019806 6887 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a708a9aa21ba4915b9ae3f1ff4de527c [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:18:53.020814 6887 raft_consensus.cc:504] T 00000000000000000000000000000000 P a708a9aa21ba4915b9ae3f1ff4de527c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a708a9aa21ba4915b9ae3f1ff4de527c" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 36755 } }
I20260809 02:18:53.021136 6891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:53.007675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33552 (local address 127.15.75.190:36755)
0809 02:18:53.008096 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:53.008112 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:53.008147 (+ 35us) server_negotiation.cc:407] Connection header received
0809 02:18:53.008862 (+ 715us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:53.008882 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:53.009203 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:53.009481 (+ 278us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:53.010346 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:53.011324 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:53.012475 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:53.013054 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:53.013175 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:53.016026 (+ 2851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:53.016031 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:53.016044 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:18:53.016071 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:53.017916 (+ 1845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:53.018397 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:53.018402 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:53.018407 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:53.018516 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:53.018812 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:53.018816 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:53.018818 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:53.019383 (+ 565us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:53.019591 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:53.019911 (+ 320us) server_negotiation.cc:299] Negotiation successful
0809 02:18:53.020200 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":151,"threads_started":1}
I20260809 02:18:53.021145 6887 leader_election.cc:302] T 00000000000000000000000000000000 P a708a9aa21ba4915b9ae3f1ff4de527c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a708a9aa21ba4915b9ae3f1ff4de527c; no voters:
I20260809 02:18:53.021536 6887 leader_election.cc:288] T 00000000000000000000000000000000 P a708a9aa21ba4915b9ae3f1ff4de527c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:18:53.021646 6893 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a708a9aa21ba4915b9ae3f1ff4de527c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:18:53.021863 6893 raft_consensus.cc:686] T 00000000000000000000000000000000 P a708a9aa21ba4915b9ae3f1ff4de527c [term 1 LEADER]: Becoming Leader. State: Replica: a708a9aa21ba4915b9ae3f1ff4de527c, State: Running, Role: LEADER
I20260809 02:18:53.022222 6893 consensus_queue.cc:237] T 00000000000000000000000000000000 P a708a9aa21ba4915b9ae3f1ff4de527c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a708a9aa21ba4915b9ae3f1ff4de527c" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 36755 } }
I20260809 02:18:53.022799 6887 sys_catalog.cc:564] T 00000000000000000000000000000000 P a708a9aa21ba4915b9ae3f1ff4de527c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:18:53.023839 6895 sys_catalog.cc:455] T 00000000000000000000000000000000 P a708a9aa21ba4915b9ae3f1ff4de527c [sys.catalog]: SysCatalogTable state changed. Reason: New leader a708a9aa21ba4915b9ae3f1ff4de527c. Latest consensus state: current_term: 1 leader_uuid: "a708a9aa21ba4915b9ae3f1ff4de527c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a708a9aa21ba4915b9ae3f1ff4de527c" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 36755 } } }
I20260809 02:18:53.023998 6895 sys_catalog.cc:458] T 00000000000000000000000000000000 P a708a9aa21ba4915b9ae3f1ff4de527c [sys.catalog]: This master's current role is: LEADER
I20260809 02:18:53.024330 6894 sys_catalog.cc:455] T 00000000000000000000000000000000 P a708a9aa21ba4915b9ae3f1ff4de527c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a708a9aa21ba4915b9ae3f1ff4de527c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a708a9aa21ba4915b9ae3f1ff4de527c" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 36755 } } }
I20260809 02:18:53.024422 6894 sys_catalog.cc:458] T 00000000000000000000000000000000 P a708a9aa21ba4915b9ae3f1ff4de527c [sys.catalog]: This master's current role is: LEADER
I20260809 02:18:53.024411 6904 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:18:53.027109 6904 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:18:53.032500 6904 catalog_manager.cc:1347] Generated new cluster ID: 60c1e4383e294c28931f3fe91c6df02b
I20260809 02:18:53.032562 6904 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:18:53.042001 6904 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:18:53.042915 6904 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:18:53.046914 6904 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a708a9aa21ba4915b9ae3f1ff4de527c: Generated new TSK 0
I20260809 02:18:53.047542 6904 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.75.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab.
Entry for principal kudu/127.15.75.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab.
I20260809 02:18:53.080084 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.129:0
--local_ip_for_outbound_sockets=127.15.75.129
--webserver_interface=127.15.75.129
--webserver_port=0
--tserver_master_addrs=127.15.75.190:36755
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab
--principal=kudu/127.15.75.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:45655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:18:53.189877 6919 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:18:53.190135 6919 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:18:53.190209 6919 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:18:53.194149 6919 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:18:53.194224 6919 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:18:53.194314 6919 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.129
I20260809 02:18:53.199338 6919 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:45655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab
--principal=kudu/127.15.75.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.75.129
--webserver_port=0
--tserver_master_addrs=127.15.75.190:36755
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6919
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.129
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:18:53.200281 6919 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:18:53.201261 6919 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:18:53.210572 6931 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:53.210589 6926 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:53.210572 6927 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:53.210805 6919 server_base.cc:1034] running on GCE node
I20260809 02:18:53.211287 6919 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:18:53.211897 6919 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:18:53.213088 6919 hybrid_clock.cc:648] HybridClock initialized: now 1786241933213078 us; error 28 us; skew 500 ppm
Aug 09 02:18:53 dist-test-slave-s11t krb5kdc[6753](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241933, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:18:53.215817 6919 init.cc:377] Logged in from keytab as kudu/127.15.75.129@KRBTEST.COM (short username kudu)
I20260809 02:18:53.216614 6919 webserver.cc:460] Webserver started at http://127.15.75.129:38441/ using document root <none> and password file <none>
I20260809 02:18:53.217175 6919 fs_manager.cc:359] Metadata directory not provided
I20260809 02:18:53.217281 6919 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:18:53.217512 6919 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:18:53.219409 6919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-0/data/instance:
uuid: "8503d46e570848f0af6333917d9fee07"
format_stamp: "Formatted at 2026-08-09 02:18:53 on dist-test-slave-s11t"
server_key: "08b453b8585072939f4b0089a3953574"
server_key_iv: "2f4271788fd852c24df45b4202c41058"
server_key_version: "encryptionkey@0"
I20260809 02:18:53.219972 6919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-0/wal/instance:
uuid: "8503d46e570848f0af6333917d9fee07"
format_stamp: "Formatted at 2026-08-09 02:18:53 on dist-test-slave-s11t"
server_key: "08b453b8585072939f4b0089a3953574"
server_key_iv: "2f4271788fd852c24df45b4202c41058"
server_key_version: "encryptionkey@0"
I20260809 02:18:53.223675 6919 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260809 02:18:53.226038 6942 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:53.227128 6919 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 02:18:53.227258 6919 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-0/wal
uuid: "8503d46e570848f0af6333917d9fee07"
format_stamp: "Formatted at 2026-08-09 02:18:53 on dist-test-slave-s11t"
server_key: "08b453b8585072939f4b0089a3953574"
server_key_iv: "2f4271788fd852c24df45b4202c41058"
server_key_version: "encryptionkey@0"
I20260809 02:18:53.227365 6919 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:18:53.239108 6919 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:18:53.242087 6919 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:18:53.242274 6919 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:18:53.242813 6919 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:18:53.243633 6919 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:18:53.243695 6919 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:53.243743 6919 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:18:53.243774 6919 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:53.252547 6919 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.129:40471
I20260809 02:18:53.252611 7120 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.129:40471 every 8 connection(s)
I20260809 02:18:53.253376 6919 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
I20260809 02:18:53.256585 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 6919
I20260809 02:18:53.256731 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-0/wal/instance
I20260809 02:18:53.257040 15662 external_mini_cluster.cc:1428] Setting key 229e7992727a58b9b5612aa389bf1f5e
Aug 09 02:18:53 dist-test-slave-s11t krb5kdc[6753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241933, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.129@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:18:53.266906 6891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:53.255010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.129:43065 (local address 127.15.75.190:36755)
0809 02:18:53.255152 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:53.255157 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:53.255948 (+ 791us) server_negotiation.cc:407] Connection header received
0809 02:18:53.257357 (+ 1409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:53.257361 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:53.257420 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:53.257500 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:53.258754 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:53.259300 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:53.260405 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:53.260650 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:53.260806 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:53.263380 (+ 2574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:53.263384 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:53.263387 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:18:53.263418 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:53.264909 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:53.265401 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:53.265406 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:53.265411 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:53.265505 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:53.265839 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:53.265844 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:53.265845 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:53.266032 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:53.266140 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:53.266598 (+ 458us) server_negotiation.cc:299] Negotiation successful
0809 02:18:53.266724 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
WARNING: no policy specified for kudu/127.15.75.130@KRBTEST.COM; defaulting to no policy
I20260809 02:18:53.267727 7126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:53.255393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36755 (local address 127.15.75.129:43065)
0809 02:18:53.255803 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:53.255843 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:53.257161 (+ 1318us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:53.257655 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:53.257662 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:53.258127 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:18:53.258614 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:53.258627 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:53.259433 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:53.259437 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:53.260283 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:53.260292 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:53.260841 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:53.260846 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:53.261023 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:53.261575 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:18:53.261590 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:18:53.263220 (+ 1630us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:18:53.265022 (+ 1802us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:18:53.265028 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:18:53.265041 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:18:53.265297 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:18:53.265603 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:18:53.265606 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:18:53.265608 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:18:53.265741 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:18:53.266139 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:18:53.266145 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:18:53.266369 (+ 224us) client_negotiation.cc:769] Sending connection context
0809 02:18:53.266557 (+ 188us) client_negotiation.cc:240] Negotiation successful
0809 02:18:53.266846 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":107,"threads_started":1}
I20260809 02:18:53.269002 7123 heartbeater.cc:347] Connected to a master server at 127.15.75.190:36755
I20260809 02:18:53.269309 7123 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.130@KRBTEST.COM" created.
I20260809 02:18:53.269918 7123 heartbeater.cc:511] Master 127.15.75.190:36755 requested a full tablet report, sending...
I20260809 02:18:53.271425 6821 ts_manager.cc:194] Registered new tserver with Master: 8503d46e570848f0af6333917d9fee07 (127.15.75.129:40471)
I20260809 02:18:53.272687 6821 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.75.129@KRBTEST.COM'} at 127.15.75.129:43065
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab.
Entry for principal kudu/127.15.75.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab.
I20260809 02:18:53.285661 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.130:0
--local_ip_for_outbound_sockets=127.15.75.130
--webserver_interface=127.15.75.130
--webserver_port=0
--tserver_master_addrs=127.15.75.190:36755
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab
--principal=kudu/127.15.75.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:45655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:18:53.398669 7131 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:18:53.398880 7131 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:18:53.398917 7131 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:18:53.402931 7131 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:18:53.402980 7131 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:18:53.403059 7131 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.130
I20260809 02:18:53.408290 7131 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:45655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab
--principal=kudu/127.15.75.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.75.130
--webserver_port=0
--tserver_master_addrs=127.15.75.190:36755
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7131
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.130
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:18:53.409263 7131 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:18:53.410207 7131 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:18:53.419907 7140 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:53.420147 7131 server_base.cc:1034] running on GCE node
W20260809 02:18:53.419932 7143 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:53.419932 7139 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:53.420817 7131 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:18:53.421550 7131 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:18:53.422760 7131 hybrid_clock.cc:648] HybridClock initialized: now 1786241933422737 us; error 43 us; skew 500 ppm
Aug 09 02:18:53 dist-test-slave-s11t krb5kdc[6753](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241933, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:18:53.426160 7131 init.cc:377] Logged in from keytab as kudu/127.15.75.130@KRBTEST.COM (short username kudu)
I20260809 02:18:53.427403 7131 webserver.cc:460] Webserver started at http://127.15.75.130:43373/ using document root <none> and password file <none>
I20260809 02:18:53.428171 7131 fs_manager.cc:359] Metadata directory not provided
I20260809 02:18:53.428282 7131 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:18:53.428539 7131 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:18:53.431609 7131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-1/data/instance:
uuid: "29341fbfcc86453da034b2c10e418b3e"
format_stamp: "Formatted at 2026-08-09 02:18:53 on dist-test-slave-s11t"
server_key: "2f512ec72d10f1d90b5bfcd88748de8d"
server_key_iv: "b6d6f1df53f3364551c64d3ec5f66f4e"
server_key_version: "encryptionkey@0"
I20260809 02:18:53.432379 7131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-1/wal/instance:
uuid: "29341fbfcc86453da034b2c10e418b3e"
format_stamp: "Formatted at 2026-08-09 02:18:53 on dist-test-slave-s11t"
server_key: "2f512ec72d10f1d90b5bfcd88748de8d"
server_key_iv: "b6d6f1df53f3364551c64d3ec5f66f4e"
server_key_version: "encryptionkey@0"
I20260809 02:18:53.437747 7131 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 02:18:53.441177 7155 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:53.442839 7131 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 02:18:53.442981 7131 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-1/wal
uuid: "29341fbfcc86453da034b2c10e418b3e"
format_stamp: "Formatted at 2026-08-09 02:18:53 on dist-test-slave-s11t"
server_key: "2f512ec72d10f1d90b5bfcd88748de8d"
server_key_iv: "b6d6f1df53f3364551c64d3ec5f66f4e"
server_key_version: "encryptionkey@0"
I20260809 02:18:53.443114 7131 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:18:53.460182 7131 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:18:53.463393 7131 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:18:53.463590 7131 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:18:53.464205 7131 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:18:53.465114 7131 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:18:53.465162 7131 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:53.465222 7131 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:18:53.465255 7131 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:53.476312 7131 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.130:45381
I20260809 02:18:53.476390 7322 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.130:45381 every 8 connection(s)
I20260809 02:18:53.477273 7131 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
I20260809 02:18:53.483448 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 7131
I20260809 02:18:53.483573 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-1/wal/instance
I20260809 02:18:53.483909 15662 external_mini_cluster.cc:1428] Setting key 057b04ed073adbf32171d6f2ad62f4a7
Aug 09 02:18:53 dist-test-slave-s11t krb5kdc[6753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241933, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.130@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:18:53.491696 6891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:53.479147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.130:44079 (local address 127.15.75.190:36755)
0809 02:18:53.479280 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:53.479284 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:53.480220 (+ 936us) server_negotiation.cc:407] Connection header received
0809 02:18:53.481654 (+ 1434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:53.481657 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:53.481715 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:53.481821 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:53.482938 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:53.483485 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:53.484994 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:53.485233 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:53.485350 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:53.487978 (+ 2628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:53.487983 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:53.487987 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:18:53.488022 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:53.489631 (+ 1609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:53.490146 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:53.490150 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:53.490152 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:53.490238 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:53.490639 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:53.490643 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:53.490645 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:53.490827 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:53.490941 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:53.491414 (+ 473us) server_negotiation.cc:299] Negotiation successful
0809 02:18:53.491543 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 02:18:53.492483 7327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:53.479619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36755 (local address 127.15.75.130:44079)
0809 02:18:53.480078 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:53.480119 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:53.481448 (+ 1329us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:53.481987 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:53.481995 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:53.482326 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:18:53.482799 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:53.482813 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:53.483676 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:53.483680 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:53.484857 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:53.484867 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:53.485389 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:53.485394 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:53.485560 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:53.486128 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:18:53.486144 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:18:53.487814 (+ 1670us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:18:53.489757 (+ 1943us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:18:53.489764 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:18:53.489786 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:18:53.490037 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:18:53.490333 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:18:53.490337 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:18:53.490339 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:18:53.490525 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:18:53.490951 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:18:53.490960 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:18:53.491185 (+ 225us) client_negotiation.cc:769] Sending connection context
0809 02:18:53.491369 (+ 184us) client_negotiation.cc:240] Negotiation successful
0809 02:18:53.491741 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":126,"threads_started":1}
I20260809 02:18:53.493673 7324 heartbeater.cc:347] Connected to a master server at 127.15.75.190:36755
I20260809 02:18:53.493932 7324 heartbeater.cc:464] Registering TS with master...
I20260809 02:18:53.494511 7324 heartbeater.cc:511] Master 127.15.75.190:36755 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.15.75.131@KRBTEST.COM; defaulting to no policy
I20260809 02:18:53.495569 6821 ts_manager.cc:194] Registered new tserver with Master: 29341fbfcc86453da034b2c10e418b3e (127.15.75.130:45381)
I20260809 02:18:53.496125 6821 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.75.130@KRBTEST.COM'} at 127.15.75.130:44079
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab.
Entry for principal kudu/127.15.75.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab.
I20260809 02:18:53.511494 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.131:0
--local_ip_for_outbound_sockets=127.15.75.131
--webserver_interface=127.15.75.131
--webserver_port=0
--tserver_master_addrs=127.15.75.190:36755
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab
--principal=kudu/127.15.75.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:45655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:18:53.620936 7330 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:18:53.621181 7330 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:18:53.621248 7330 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:18:53.625507 7330 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:18:53.625578 7330 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:18:53.625661 7330 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.131
I20260809 02:18:53.631042 7330 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:45655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab
--principal=kudu/127.15.75.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.75.131
--webserver_port=0
--tserver_master_addrs=127.15.75.190:36755
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7330
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.131
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:18:53.632063 7330 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:18:53.633029 7330 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:18:53.642386 7337 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:53.642504 7330 server_base.cc:1034] running on GCE node
W20260809 02:18:53.642457 7336 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:53.642447 7340 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:53.642985 7330 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:18:53.643558 7330 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:18:53.644755 7330 hybrid_clock.cc:648] HybridClock initialized: now 1786241933644735 us; error 31 us; skew 500 ppm
Aug 09 02:18:53 dist-test-slave-s11t krb5kdc[6753](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241933, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:18:53.647353 7330 init.cc:377] Logged in from keytab as kudu/127.15.75.131@KRBTEST.COM (short username kudu)
I20260809 02:18:53.648221 7330 webserver.cc:460] Webserver started at http://127.15.75.131:36691/ using document root <none> and password file <none>
I20260809 02:18:53.648821 7330 fs_manager.cc:359] Metadata directory not provided
I20260809 02:18:53.648911 7330 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:18:53.649113 7330 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:18:53.651050 7330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-2/data/instance:
uuid: "4ba2d99b0bd74c71b736bed8f281e453"
format_stamp: "Formatted at 2026-08-09 02:18:53 on dist-test-slave-s11t"
server_key: "2d5bef98351dd6812a39e5b7901a1654"
server_key_iv: "9baa0bb5e138d4a88823d87de26ce886"
server_key_version: "encryptionkey@0"
I20260809 02:18:53.651602 7330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-2/wal/instance:
uuid: "4ba2d99b0bd74c71b736bed8f281e453"
format_stamp: "Formatted at 2026-08-09 02:18:53 on dist-test-slave-s11t"
server_key: "2d5bef98351dd6812a39e5b7901a1654"
server_key_iv: "9baa0bb5e138d4a88823d87de26ce886"
server_key_version: "encryptionkey@0"
I20260809 02:18:53.655107 7330 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260809 02:18:53.657440 7350 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:53.658566 7330 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 02:18:53.658699 7330 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-2/wal
uuid: "4ba2d99b0bd74c71b736bed8f281e453"
format_stamp: "Formatted at 2026-08-09 02:18:53 on dist-test-slave-s11t"
server_key: "2d5bef98351dd6812a39e5b7901a1654"
server_key_iv: "9baa0bb5e138d4a88823d87de26ce886"
server_key_version: "encryptionkey@0"
I20260809 02:18:53.658808 7330 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:18:53.693389 7330 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:18:53.696254 7330 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:18:53.696466 7330 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:18:53.697082 7330 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:18:53.697956 7330 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:18:53.698028 7330 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:53.698101 7330 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:18:53.698156 7330 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:53.707233 7330 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.131:45603
I20260809 02:18:53.707265 7517 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.131:45603 every 8 connection(s)
I20260809 02:18:53.708137 7330 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
I20260809 02:18:53.717877 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 7330
I20260809 02:18:53.718014 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-2/wal/instance
Aug 09 02:18:53 dist-test-slave-s11t krb5kdc[6753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241933, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.131@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:18:53.718326 15662 external_mini_cluster.cc:1428] Setting key 0771c5b21f37fcab0013cf9dba303c7e
I20260809 02:18:53.722285 6891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:53.709917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.131:46907 (local address 127.15.75.190:36755)
0809 02:18:53.710063 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:53.710067 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:53.710866 (+ 799us) server_negotiation.cc:407] Connection header received
0809 02:18:53.712202 (+ 1336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:53.712205 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:53.712265 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:53.712375 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:53.713520 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:53.714056 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:53.715182 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:53.715570 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:53.715677 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:53.718802 (+ 3125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:53.718805 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:53.718808 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:18:53.718834 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:53.720485 (+ 1651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:53.721033 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:53.721035 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:53.721036 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:53.721110 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:53.721414 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:53.721417 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:53.721418 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:53.721576 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:53.721647 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:53.722079 (+ 432us) server_negotiation.cc:299] Negotiation successful
0809 02:18:53.722171 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 02:18:53.723059 7521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:53.710270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36755 (local address 127.15.75.131:46907)
0809 02:18:53.710717 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:53.710759 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:53.711994 (+ 1235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:53.712543 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:53.712550 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:53.712923 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:18:53.713373 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:53.713385 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:53.714181 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:53.714190 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:53.715057 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:53.715066 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:53.715753 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:53.715760 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:53.716034 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:53.716767 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:18:53.716790 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:18:53.718643 (+ 1853us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:18:53.720599 (+ 1956us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:18:53.720608 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:18:53.720622 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:18:53.720943 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:18:53.721190 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:18:53.721192 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:18:53.721193 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:18:53.721317 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:18:53.721659 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:18:53.721664 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:18:53.721867 (+ 203us) client_negotiation.cc:769] Sending connection context
0809 02:18:53.722017 (+ 150us) client_negotiation.cc:240] Negotiation successful
0809 02:18:53.722245 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":114,"threads_started":1}
I20260809 02:18:53.724083 7519 heartbeater.cc:347] Connected to a master server at 127.15.75.190:36755
I20260809 02:18:53.724310 7519 heartbeater.cc:464] Registering TS with master...
I20260809 02:18:53.724876 7519 heartbeater.cc:511] Master 127.15.75.190:36755 requested a full tablet report, sending...
I20260809 02:18:53.725829 6821 ts_manager.cc:194] Registered new tserver with Master: 4ba2d99b0bd74c71b736bed8f281e453 (127.15.75.131:45603)
I20260809 02:18:53.726394 6821 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.75.131@KRBTEST.COM'} at 127.15.75.131:46907
I20260809 02:18:53.732091 15662 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 02:18:53.740636 6891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:53.734097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33566 (local address 127.15.75.190:36755)
0809 02:18:53.734262 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:53.734266 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:53.734379 (+ 113us) server_negotiation.cc:407] Connection header received
0809 02:18:53.734466 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:53.734468 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:53.734502 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:53.734572 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:53.735180 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:53.735657 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:53.736418 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:53.736617 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:53.736751 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:53.737915 (+ 1164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:53.737918 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:53.737920 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:18:53.737941 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:53.739312 (+ 1371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:53.739660 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:53.739662 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:53.739663 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:53.739734 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:53.739967 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:53.739969 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:53.739970 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:53.740159 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:53.740261 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:53.740446 (+ 185us) server_negotiation.cc:299] Negotiation successful
0809 02:18:53.740533 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260809 02:18:53.743255 6821 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33566:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:18:53.746208 6821 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:18:53.760344 7540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:53.755009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.129:40471 (local address 127.0.0.1:60050)
0809 02:18:53.755273 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:53.755288 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:53.755453 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:53.756293 (+ 840us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:53.756296 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:53.756313 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:18:53.756419 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:53.756425 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:53.757605 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:53.757611 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:53.758883 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:53.758899 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:53.759799 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:53.759802 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:53.759907 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:53.759948 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 02:18:53.760063 (+ 115us) client_negotiation.cc:240] Negotiation successful
0809 02:18:53.760177 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"mutex_wait_us":47,"spinlock_wait_cycles":96384,"thread_start_us":102,"threads_started":1}
I20260809 02:18:53.760360 7541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:53.755155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60050 (local address 127.15.75.129:40471)
0809 02:18:53.755832 (+ 677us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:53.755848 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:53.755872 (+ 24us) server_negotiation.cc:407] Connection header received
0809 02:18:53.755951 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:53.755956 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:53.756134 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:53.756565 (+ 431us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:18:53.756764 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:53.757472 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:53.759019 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:53.759673 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:53.759785 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:53.759921 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:53.760110 (+ 189us) server_negotiation.cc:299] Negotiation successful
0809 02:18:53.760208 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":569,"thread_start_us":76,"threads_started":1}
I20260809 02:18:53.760972 7537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:53.754349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.130:45381 (local address 127.0.0.1:36990)
0809 02:18:53.754796 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:53.754841 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:53.755456 (+ 615us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:53.756232 (+ 776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:53.756238 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:53.756259 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:18:53.756391 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:53.756407 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:53.757911 (+ 1504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:53.757914 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:53.759383 (+ 1469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:53.759400 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:53.760503 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:53.760507 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:53.760639 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:53.760661 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 02:18:53.760750 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 02:18:53.760824 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"mutex_wait_us":188,"spinlock_wait_cycles":1013248,"thread_start_us":65,"threads_started":1}
I20260809 02:18:53.761113 7538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:53.754483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36990 (local address 127.15.75.130:45381)
0809 02:18:53.755832 (+ 1349us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:53.755851 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:53.755870 (+ 19us) server_negotiation.cc:407] Connection header received
0809 02:18:53.755935 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:53.755939 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:53.756083 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:53.756241 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:18:53.756517 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:53.757746 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:53.759522 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:53.760349 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:53.760631 (+ 282us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:53.760795 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:53.760844 (+ 49us) server_negotiation.cc:299] Negotiation successful
0809 02:18:53.760937 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1243,"thread_start_us":96,"threads_started":1}
I20260809 02:18:53.761227 7539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:53.754490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.131:45603 (local address 127.0.0.1:42542)
0809 02:18:53.754835 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:53.754851 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:53.755593 (+ 742us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:53.756428 (+ 835us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:53.756431 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:53.756447 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:18:53.756548 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:53.756554 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:53.757961 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:53.757964 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:53.759383 (+ 1419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:53.759395 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:53.760571 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:53.760574 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:53.760718 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:53.760747 (+ 29us) client_negotiation.cc:769] Sending connection context
0809 02:18:53.760812 (+ 65us) client_negotiation.cc:240] Negotiation successful
0809 02:18:53.760888 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"mutex_wait_us":130,"spinlock_wait_cycles":4864,"thread_start_us":105,"threads_started":1}
I20260809 02:18:53.761768 7543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:53.755524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42542 (local address 127.15.75.131:45603)
0809 02:18:53.755878 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:53.755901 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:53.755920 (+ 19us) server_negotiation.cc:407] Connection header received
0809 02:18:53.755980 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:53.755984 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:53.756134 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:53.756284 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:18:53.756664 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:53.757807 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:53.759535 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:53.760267 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:53.760374 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:53.761504 (+ 1130us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:53.761556 (+ 52us) server_negotiation.cc:299] Negotiation successful
0809 02:18:53.761637 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":131,"threads_started":1}
I20260809 02:18:53.763702 7025 tablet_service.cc:1467] Processing CreateTablet for tablet 3be740b2f89a415f97a0ab7110c7b18f (DEFAULT_TABLE table=test-table [id=939fbd64d4634ff6a99314e90670f7c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:18:53.764174 7419 tablet_service.cc:1467] Processing CreateTablet for tablet 3be740b2f89a415f97a0ab7110c7b18f (DEFAULT_TABLE table=test-table [id=939fbd64d4634ff6a99314e90670f7c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:18:53.764771 7025 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3be740b2f89a415f97a0ab7110c7b18f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:18:53.765105 7419 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3be740b2f89a415f97a0ab7110c7b18f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:18:53.766518 7225 tablet_service.cc:1467] Processing CreateTablet for tablet 3be740b2f89a415f97a0ab7110c7b18f (DEFAULT_TABLE table=test-table [id=939fbd64d4634ff6a99314e90670f7c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:18:53.767447 7225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3be740b2f89a415f97a0ab7110c7b18f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:18:53.771341 7544 tablet_bootstrap.cc:492] T 3be740b2f89a415f97a0ab7110c7b18f P 4ba2d99b0bd74c71b736bed8f281e453: Bootstrap starting.
I20260809 02:18:53.773010 7547 tablet_bootstrap.cc:492] T 3be740b2f89a415f97a0ab7110c7b18f P 8503d46e570848f0af6333917d9fee07: Bootstrap starting.
I20260809 02:18:53.773748 7544 tablet_bootstrap.cc:654] T 3be740b2f89a415f97a0ab7110c7b18f P 4ba2d99b0bd74c71b736bed8f281e453: Neither blocks nor log segments found. Creating new log.
I20260809 02:18:53.774756 7548 tablet_bootstrap.cc:492] T 3be740b2f89a415f97a0ab7110c7b18f P 29341fbfcc86453da034b2c10e418b3e: Bootstrap starting.
I20260809 02:18:53.775000 7544 log.cc:824] T 3be740b2f89a415f97a0ab7110c7b18f P 4ba2d99b0bd74c71b736bed8f281e453: Log is configured to *not* fsync() on all Append() calls
I20260809 02:18:53.775110 7547 tablet_bootstrap.cc:654] T 3be740b2f89a415f97a0ab7110c7b18f P 8503d46e570848f0af6333917d9fee07: Neither blocks nor log segments found. Creating new log.
I20260809 02:18:53.775998 7547 log.cc:824] T 3be740b2f89a415f97a0ab7110c7b18f P 8503d46e570848f0af6333917d9fee07: Log is configured to *not* fsync() on all Append() calls
I20260809 02:18:53.777170 7548 tablet_bootstrap.cc:654] T 3be740b2f89a415f97a0ab7110c7b18f P 29341fbfcc86453da034b2c10e418b3e: Neither blocks nor log segments found. Creating new log.
I20260809 02:18:53.777472 7544 tablet_bootstrap.cc:492] T 3be740b2f89a415f97a0ab7110c7b18f P 4ba2d99b0bd74c71b736bed8f281e453: No bootstrap required, opened a new log
I20260809 02:18:53.777664 7544 ts_tablet_manager.cc:1397] T 3be740b2f89a415f97a0ab7110c7b18f P 4ba2d99b0bd74c71b736bed8f281e453: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260809 02:18:53.777930 7547 tablet_bootstrap.cc:492] T 3be740b2f89a415f97a0ab7110c7b18f P 8503d46e570848f0af6333917d9fee07: No bootstrap required, opened a new log
I20260809 02:18:53.777936 7548 log.cc:824] T 3be740b2f89a415f97a0ab7110c7b18f P 29341fbfcc86453da034b2c10e418b3e: Log is configured to *not* fsync() on all Append() calls
I20260809 02:18:53.778102 7547 ts_tablet_manager.cc:1397] T 3be740b2f89a415f97a0ab7110c7b18f P 8503d46e570848f0af6333917d9fee07: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 02:18:53.779596 7548 tablet_bootstrap.cc:492] T 3be740b2f89a415f97a0ab7110c7b18f P 29341fbfcc86453da034b2c10e418b3e: No bootstrap required, opened a new log
I20260809 02:18:53.779778 7548 ts_tablet_manager.cc:1397] T 3be740b2f89a415f97a0ab7110c7b18f P 29341fbfcc86453da034b2c10e418b3e: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260809 02:18:53.781788 7544 raft_consensus.cc:348] T 3be740b2f89a415f97a0ab7110c7b18f P 4ba2d99b0bd74c71b736bed8f281e453 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29341fbfcc86453da034b2c10e418b3e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45381 } } peers { permanent_uuid: "4ba2d99b0bd74c71b736bed8f281e453" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45603 } } peers { permanent_uuid: "8503d46e570848f0af6333917d9fee07" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 40471 } }
I20260809 02:18:53.782066 7544 raft_consensus.cc:374] T 3be740b2f89a415f97a0ab7110c7b18f P 4ba2d99b0bd74c71b736bed8f281e453 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:18:53.782135 7544 raft_consensus.cc:729] T 3be740b2f89a415f97a0ab7110c7b18f P 4ba2d99b0bd74c71b736bed8f281e453 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ba2d99b0bd74c71b736bed8f281e453, State: Initialized, Role: FOLLOWER
I20260809 02:18:53.782272 7547 raft_consensus.cc:348] T 3be740b2f89a415f97a0ab7110c7b18f P 8503d46e570848f0af6333917d9fee07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29341fbfcc86453da034b2c10e418b3e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45381 } } peers { permanent_uuid: "4ba2d99b0bd74c71b736bed8f281e453" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45603 } } peers { permanent_uuid: "8503d46e570848f0af6333917d9fee07" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 40471 } }
I20260809 02:18:53.782501 7547 raft_consensus.cc:374] T 3be740b2f89a415f97a0ab7110c7b18f P 8503d46e570848f0af6333917d9fee07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:18:53.782565 7547 raft_consensus.cc:729] T 3be740b2f89a415f97a0ab7110c7b18f P 8503d46e570848f0af6333917d9fee07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8503d46e570848f0af6333917d9fee07, State: Initialized, Role: FOLLOWER
I20260809 02:18:53.782529 7548 raft_consensus.cc:348] T 3be740b2f89a415f97a0ab7110c7b18f P 29341fbfcc86453da034b2c10e418b3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29341fbfcc86453da034b2c10e418b3e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45381 } } peers { permanent_uuid: "4ba2d99b0bd74c71b736bed8f281e453" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45603 } } peers { permanent_uuid: "8503d46e570848f0af6333917d9fee07" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 40471 } }
I20260809 02:18:53.782799 7548 raft_consensus.cc:374] T 3be740b2f89a415f97a0ab7110c7b18f P 29341fbfcc86453da034b2c10e418b3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:18:53.782852 7548 raft_consensus.cc:729] T 3be740b2f89a415f97a0ab7110c7b18f P 29341fbfcc86453da034b2c10e418b3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29341fbfcc86453da034b2c10e418b3e, State: Initialized, Role: FOLLOWER
I20260809 02:18:53.782759 7544 consensus_queue.cc:260] T 3be740b2f89a415f97a0ab7110c7b18f P 4ba2d99b0bd74c71b736bed8f281e453 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29341fbfcc86453da034b2c10e418b3e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45381 } } peers { permanent_uuid: "4ba2d99b0bd74c71b736bed8f281e453" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45603 } } peers { permanent_uuid: "8503d46e570848f0af6333917d9fee07" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 40471 } }
I20260809 02:18:53.783020 7547 consensus_queue.cc:260] T 3be740b2f89a415f97a0ab7110c7b18f P 8503d46e570848f0af6333917d9fee07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29341fbfcc86453da034b2c10e418b3e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45381 } } peers { permanent_uuid: "4ba2d99b0bd74c71b736bed8f281e453" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45603 } } peers { permanent_uuid: "8503d46e570848f0af6333917d9fee07" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 40471 } }
I20260809 02:18:53.783321 7548 consensus_queue.cc:260] T 3be740b2f89a415f97a0ab7110c7b18f P 29341fbfcc86453da034b2c10e418b3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29341fbfcc86453da034b2c10e418b3e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45381 } } peers { permanent_uuid: "4ba2d99b0bd74c71b736bed8f281e453" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45603 } } peers { permanent_uuid: "8503d46e570848f0af6333917d9fee07" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 40471 } }
I20260809 02:18:53.783946 7544 ts_tablet_manager.cc:1428] T 3be740b2f89a415f97a0ab7110c7b18f P 4ba2d99b0bd74c71b736bed8f281e453: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 02:18:53.783947 7547 ts_tablet_manager.cc:1428] T 3be740b2f89a415f97a0ab7110c7b18f P 8503d46e570848f0af6333917d9fee07: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260809 02:18:53.783969 7324 heartbeater.cc:503] Master 127.15.75.190:36755 was elected leader, sending a full tablet report...
I20260809 02:18:53.784263 7123 heartbeater.cc:503] Master 127.15.75.190:36755 was elected leader, sending a full tablet report...
I20260809 02:18:53.784267 7519 heartbeater.cc:503] Master 127.15.75.190:36755 was elected leader, sending a full tablet report...
I20260809 02:18:53.785250 7548 ts_tablet_manager.cc:1428] T 3be740b2f89a415f97a0ab7110c7b18f P 29341fbfcc86453da034b2c10e418b3e: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260809 02:18:53.854413 7553 raft_consensus.cc:482] T 3be740b2f89a415f97a0ab7110c7b18f P 8503d46e570848f0af6333917d9fee07 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:18:53.854641 7553 raft_consensus.cc:504] T 3be740b2f89a415f97a0ab7110c7b18f P 8503d46e570848f0af6333917d9fee07 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29341fbfcc86453da034b2c10e418b3e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45381 } } peers { permanent_uuid: "4ba2d99b0bd74c71b736bed8f281e453" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45603 } } peers { permanent_uuid: "8503d46e570848f0af6333917d9fee07" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 40471 } }
I20260809 02:18:53.855748 7553 leader_election.cc:288] T 3be740b2f89a415f97a0ab7110c7b18f P 8503d46e570848f0af6333917d9fee07 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29341fbfcc86453da034b2c10e418b3e (127.15.75.130:45381), 4ba2d99b0bd74c71b736bed8f281e453 (127.15.75.131:45603)
I20260809 02:18:53.859877 7557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:53.856027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.130:45381 (local address 127.15.75.129:49241)
0809 02:18:53.856476 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:53.856492 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:53.856603 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:53.856950 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:53.856953 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:53.856968 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:18:53.857092 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:53.857098 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:53.857829 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:53.857832 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:53.858765 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:53.858774 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:53.859426 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:53.859430 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:53.859539 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:53.859565 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 02:18:53.859638 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 02:18:53.859711 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":375,"mutex_wait_us":54,"thread_start_us":121,"threads_started":1}
I20260809 02:18:53.859879 7538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:53.856133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.129:49241 (local address 127.15.75.130:45381)
0809 02:18:53.856328 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:53.856333 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:53.856593 (+ 260us) server_negotiation.cc:407] Connection header received
0809 02:18:53.856743 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:53.856747 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:53.856826 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:53.856925 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:18:53.857215 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:53.857688 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:53.858905 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:53.859301 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:53.859393 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:53.859431 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:53.859674 (+ 243us) server_negotiation.cc:299] Negotiation successful
0809 02:18:53.859747 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260809 02:18:53.860049 7555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:53.855988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.131:45603 (local address 127.15.75.129:59545)
0809 02:18:53.856401 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:53.856413 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:53.856509 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:53.856822 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:53.856825 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:53.856841 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:18:53.857002 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:53.857010 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:53.857788 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:53.857791 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:53.858775 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:53.858784 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:53.859580 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:53.859582 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:53.859702 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:53.859725 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 02:18:53.859801 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 02:18:53.859861 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"mutex_wait_us":70,"thread_start_us":139,"threads_started":1}
I20260809 02:18:53.860054 7543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:53.856178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.129:59545 (local address 127.15.75.131:45603)
0809 02:18:53.856334 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:53.856339 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:53.856502 (+ 163us) server_negotiation.cc:407] Connection header received
0809 02:18:53.856623 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:53.856630 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:53.856709 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:53.856783 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:18:53.857131 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:53.857666 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:53.859028 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:53.859458 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:53.859572 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:53.859635 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:53.859836 (+ 201us) server_negotiation.cc:299] Negotiation successful
0809 02:18:53.859931 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 02:18:53.860682 7258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3be740b2f89a415f97a0ab7110c7b18f" candidate_uuid: "8503d46e570848f0af6333917d9fee07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29341fbfcc86453da034b2c10e418b3e" is_pre_election: true
I20260809 02:18:53.860704 7448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3be740b2f89a415f97a0ab7110c7b18f" candidate_uuid: "8503d46e570848f0af6333917d9fee07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ba2d99b0bd74c71b736bed8f281e453" is_pre_election: true
I20260809 02:18:53.860997 7448 raft_consensus.cc:2454] T 3be740b2f89a415f97a0ab7110c7b18f P 4ba2d99b0bd74c71b736bed8f281e453 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8503d46e570848f0af6333917d9fee07 in term 0.
I20260809 02:18:53.860997 7258 raft_consensus.cc:2454] T 3be740b2f89a415f97a0ab7110c7b18f P 29341fbfcc86453da034b2c10e418b3e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8503d46e570848f0af6333917d9fee07 in term 0.
I20260809 02:18:53.861518 6947 leader_election.cc:302] T 3be740b2f89a415f97a0ab7110c7b18f P 8503d46e570848f0af6333917d9fee07 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 29341fbfcc86453da034b2c10e418b3e, 4ba2d99b0bd74c71b736bed8f281e453, 8503d46e570848f0af6333917d9fee07; no voters:
I20260809 02:18:53.861799 7553 raft_consensus.cc:2781] T 3be740b2f89a415f97a0ab7110c7b18f P 8503d46e570848f0af6333917d9fee07 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:18:53.861907 7553 raft_consensus.cc:482] T 3be740b2f89a415f97a0ab7110c7b18f P 8503d46e570848f0af6333917d9fee07 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:18:53.861986 7553 raft_consensus.cc:3037] T 3be740b2f89a415f97a0ab7110c7b18f P 8503d46e570848f0af6333917d9fee07 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:18:53.863101 7553 raft_consensus.cc:504] T 3be740b2f89a415f97a0ab7110c7b18f P 8503d46e570848f0af6333917d9fee07 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29341fbfcc86453da034b2c10e418b3e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45381 } } peers { permanent_uuid: "4ba2d99b0bd74c71b736bed8f281e453" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45603 } } peers { permanent_uuid: "8503d46e570848f0af6333917d9fee07" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 40471 } }
I20260809 02:18:53.863420 7553 leader_election.cc:288] T 3be740b2f89a415f97a0ab7110c7b18f P 8503d46e570848f0af6333917d9fee07 [CANDIDATE]: Term 1 election: Requested vote from peers 29341fbfcc86453da034b2c10e418b3e (127.15.75.130:45381), 4ba2d99b0bd74c71b736bed8f281e453 (127.15.75.131:45603)
I20260809 02:18:53.863776 7448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3be740b2f89a415f97a0ab7110c7b18f" candidate_uuid: "8503d46e570848f0af6333917d9fee07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ba2d99b0bd74c71b736bed8f281e453"
I20260809 02:18:53.863776 7258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3be740b2f89a415f97a0ab7110c7b18f" candidate_uuid: "8503d46e570848f0af6333917d9fee07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29341fbfcc86453da034b2c10e418b3e"
I20260809 02:18:53.863894 7258 raft_consensus.cc:3037] T 3be740b2f89a415f97a0ab7110c7b18f P 29341fbfcc86453da034b2c10e418b3e [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:18:53.863902 7448 raft_consensus.cc:3037] T 3be740b2f89a415f97a0ab7110c7b18f P 4ba2d99b0bd74c71b736bed8f281e453 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:18:53.865175 7448 raft_consensus.cc:2454] T 3be740b2f89a415f97a0ab7110c7b18f P 4ba2d99b0bd74c71b736bed8f281e453 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8503d46e570848f0af6333917d9fee07 in term 1.
I20260809 02:18:53.865175 7258 raft_consensus.cc:2454] T 3be740b2f89a415f97a0ab7110c7b18f P 29341fbfcc86453da034b2c10e418b3e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8503d46e570848f0af6333917d9fee07 in term 1.
I20260809 02:18:53.865511 6943 leader_election.cc:302] T 3be740b2f89a415f97a0ab7110c7b18f P 8503d46e570848f0af6333917d9fee07 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29341fbfcc86453da034b2c10e418b3e, 8503d46e570848f0af6333917d9fee07; no voters:
I20260809 02:18:53.865674 7553 raft_consensus.cc:2781] T 3be740b2f89a415f97a0ab7110c7b18f P 8503d46e570848f0af6333917d9fee07 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:18:53.865862 7553 raft_consensus.cc:686] T 3be740b2f89a415f97a0ab7110c7b18f P 8503d46e570848f0af6333917d9fee07 [term 1 LEADER]: Becoming Leader. State: Replica: 8503d46e570848f0af6333917d9fee07, State: Running, Role: LEADER
I20260809 02:18:53.866204 7553 consensus_queue.cc:237] T 3be740b2f89a415f97a0ab7110c7b18f P 8503d46e570848f0af6333917d9fee07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29341fbfcc86453da034b2c10e418b3e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45381 } } peers { permanent_uuid: "4ba2d99b0bd74c71b736bed8f281e453" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45603 } } peers { permanent_uuid: "8503d46e570848f0af6333917d9fee07" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 40471 } }
I20260809 02:18:53.869210 6821 catalog_manager.cc:5495] T 3be740b2f89a415f97a0ab7110c7b18f P 8503d46e570848f0af6333917d9fee07 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8503d46e570848f0af6333917d9fee07 (127.15.75.129). New cstate: current_term: 1 leader_uuid: "8503d46e570848f0af6333917d9fee07" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29341fbfcc86453da034b2c10e418b3e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ba2d99b0bd74c71b736bed8f281e453" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8503d46e570848f0af6333917d9fee07" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 40471 } health_report { overall_health: HEALTHY } } }
I20260809 02:18:53.879807 7541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:53.876751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60066 (local address 127.15.75.129:40471)
0809 02:18:53.876863 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:53.876867 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:53.876878 (+ 11us) server_negotiation.cc:407] Connection header received
0809 02:18:53.877009 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:53.877011 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:53.877057 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:53.877137 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:18:53.877317 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:53.877778 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:53.878532 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:53.878775 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:53.878850 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:53.879115 (+ 265us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:18:53.879472 (+ 357us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:18:53.879598 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:53.879680 (+ 82us) server_negotiation.cc:299] Negotiation successful
0809 02:18:53.879722 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 02:18:53.885018 7448 raft_consensus.cc:1261] T 3be740b2f89a415f97a0ab7110c7b18f P 4ba2d99b0bd74c71b736bed8f281e453 [term 1 FOLLOWER]: Refusing update from remote peer 8503d46e570848f0af6333917d9fee07: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:18:53.885046 7258 raft_consensus.cc:1261] T 3be740b2f89a415f97a0ab7110c7b18f P 29341fbfcc86453da034b2c10e418b3e [term 1 FOLLOWER]: Refusing update from remote peer 8503d46e570848f0af6333917d9fee07: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:18:53.885679 7558 consensus_queue.cc:1035] T 3be740b2f89a415f97a0ab7110c7b18f P 8503d46e570848f0af6333917d9fee07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29341fbfcc86453da034b2c10e418b3e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:18:53.885847 7553 consensus_queue.cc:1035] T 3be740b2f89a415f97a0ab7110c7b18f P 8503d46e570848f0af6333917d9fee07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ba2d99b0bd74c71b736bed8f281e453" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:18:53.892215 7563 mvcc.cc:205] Tried to move back new op lower bound from 7316446961186811904 to 7316446961118429184. Current Snapshot: MvccSnapshot[applied={T|T < 7316446961186811904}]
I20260809 02:18:53.893316 7565 mvcc.cc:205] Tried to move back new op lower bound from 7316446961186811904 to 7316446961118429184. Current Snapshot: MvccSnapshot[applied={T|T < 7316446961186811904}]
I20260809 02:18:53.901592 6818 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33566:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:18:53.901751 6818 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33566:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:18:53.904093 6818 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a708a9aa21ba4915b9ae3f1ff4de527c: Sending DeleteTablet for 3 replicas of tablet 3be740b2f89a415f97a0ab7110c7b18f
I20260809 02:18:53.904784 7225 tablet_service.cc:1514] Processing DeleteTablet for tablet 3be740b2f89a415f97a0ab7110c7b18f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:18:53 UTC) from {username='kudu', principal='kudu/127.15.75.190@KRBTEST.COM'} at 127.0.0.1:36990
I20260809 02:18:53.904920 7419 tablet_service.cc:1514] Processing DeleteTablet for tablet 3be740b2f89a415f97a0ab7110c7b18f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:18:53 UTC) from {username='kudu', principal='kudu/127.15.75.190@KRBTEST.COM'} at 127.0.0.1:42542
I20260809 02:18:53.905010 7025 tablet_service.cc:1514] Processing DeleteTablet for tablet 3be740b2f89a415f97a0ab7110c7b18f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:18:53 UTC) from {username='kudu', principal='kudu/127.15.75.190@KRBTEST.COM'} at 127.0.0.1:60050
I20260809 02:18:53.905321 7583 tablet_replica.cc:330] T 3be740b2f89a415f97a0ab7110c7b18f P 4ba2d99b0bd74c71b736bed8f281e453: stopping tablet replica
I20260809 02:18:53.905586 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 6919
I20260809 02:18:53.905594 7585 tablet_replica.cc:330] T 3be740b2f89a415f97a0ab7110c7b18f P 8503d46e570848f0af6333917d9fee07: stopping tablet replica
I20260809 02:18:53.905627 7583 raft_consensus.cc:2229] T 3be740b2f89a415f97a0ab7110c7b18f P 4ba2d99b0bd74c71b736bed8f281e453 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:18:53.905830 7585 raft_consensus.cc:2229] T 3be740b2f89a415f97a0ab7110c7b18f P 8503d46e570848f0af6333917d9fee07 [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:18:53.905890 7583 raft_consensus.cc:2258] T 3be740b2f89a415f97a0ab7110c7b18f P 4ba2d99b0bd74c71b736bed8f281e453 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:18:53.909420 7581 tablet_replica.cc:330] T 3be740b2f89a415f97a0ab7110c7b18f P 29341fbfcc86453da034b2c10e418b3e: stopping tablet replica
I20260809 02:18:53.909696 7581 raft_consensus.cc:2229] T 3be740b2f89a415f97a0ab7110c7b18f P 29341fbfcc86453da034b2c10e418b3e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:18:53.909973 7581 raft_consensus.cc:2258] T 3be740b2f89a415f97a0ab7110c7b18f P 29341fbfcc86453da034b2c10e418b3e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:18:53.910218 7583 ts_tablet_manager.cc:1905] T 3be740b2f89a415f97a0ab7110c7b18f P 4ba2d99b0bd74c71b736bed8f281e453: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:18:53.911954 7581 ts_tablet_manager.cc:1905] T 3be740b2f89a415f97a0ab7110c7b18f P 29341fbfcc86453da034b2c10e418b3e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:18:53.913033 7583 ts_tablet_manager.cc:1918] T 3be740b2f89a415f97a0ab7110c7b18f P 4ba2d99b0bd74c71b736bed8f281e453: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:18:53.913134 7583 log.cc:1196] T 3be740b2f89a415f97a0ab7110c7b18f P 4ba2d99b0bd74c71b736bed8f281e453: Deleting WAL directory at /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-2/wal/wals/3be740b2f89a415f97a0ab7110c7b18f
I20260809 02:18:53.913498 7583 ts_tablet_manager.cc:1939] T 3be740b2f89a415f97a0ab7110c7b18f P 4ba2d99b0bd74c71b736bed8f281e453: Deleting consensus metadata
I20260809 02:18:53.914640 7581 ts_tablet_manager.cc:1918] T 3be740b2f89a415f97a0ab7110c7b18f P 29341fbfcc86453da034b2c10e418b3e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:18:53.914742 7581 log.cc:1196] T 3be740b2f89a415f97a0ab7110c7b18f P 29341fbfcc86453da034b2c10e418b3e: Deleting WAL directory at /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-1/wal/wals/3be740b2f89a415f97a0ab7110c7b18f
I20260809 02:18:53.914734 6804 catalog_manager.cc:4841] TS 4ba2d99b0bd74c71b736bed8f281e453 (127.15.75.131:45603): tablet 3be740b2f89a415f97a0ab7110c7b18f (table test-table [id=939fbd64d4634ff6a99314e90670f7c7]) successfully deleted
I20260809 02:18:53.915076 7581 ts_tablet_manager.cc:1939] T 3be740b2f89a415f97a0ab7110c7b18f P 29341fbfcc86453da034b2c10e418b3e: Deleting consensus metadata
I20260809 02:18:53.915980 6801 catalog_manager.cc:4841] TS 29341fbfcc86453da034b2c10e418b3e (127.15.75.130:45381): tablet 3be740b2f89a415f97a0ab7110c7b18f (table test-table [id=939fbd64d4634ff6a99314e90670f7c7]) successfully deleted
W20260809 02:18:53.920055 6804 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.75.129:40471 (error 108)
I20260809 02:18:53.920339 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 7131
I20260809 02:18:53.921593 7539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:53.921120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.129:40471 (local address 127.0.0.1:60078)
0809 02:18:53.921376 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:53.921484 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.129:40471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147}
W20260809 02:18:53.922940 6804 catalog_manager.cc:4584] TS 8503d46e570848f0af6333917d9fee07 (127.15.75.129:40471): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3be740b2f89a415f97a0ab7110c7b18f: Network error: Client connection negotiation failed: client connection to 127.15.75.129:40471: connect: Connection refused (error 111)
I20260809 02:18:53.934027 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 7330
I20260809 02:18:53.946632 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 6775
I20260809 02:19:13.959336 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:36755
--webserver_interface=127.15.75.190
--webserver_port=36907
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab
--principal=kudu/127.15.75.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:45655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.75.190:36755
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:19:14.069650 7588 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:19:14.069928 7588 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:19:14.070006 7588 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:19:14.073889 7588 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 02:19:14.073966 7588 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 02:19:14.073993 7588 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 02:19:14.074025 7588 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:19:14.074062 7588 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:19:14.074100 7588 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:19:14.074150 7588 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:19:14.074184 7588 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:19:14.079176 7588 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:45655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.15.75.190:36755
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab
--principal=kudu/127.15.75.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:36755
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.75.190
--webserver_port=36907
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7588
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:19:14.080111 7588 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:19:14.081089 7588 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:19:14.094607 7600 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:19:14.094623 7595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:19:14.094725 7588 server_base.cc:1034] running on GCE node
W20260809 02:19:14.094907 7596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:19:14.095510 7588 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:19:14.096534 7588 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:19:14.097693 7588 hybrid_clock.cc:648] HybridClock initialized: now 1786241954097677 us; error 30 us; skew 500 ppm
Aug 09 02:19:14 dist-test-slave-s11t krb5kdc[6753](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241954, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:19:14.100356 7588 init.cc:377] Logged in from keytab as kudu/127.15.75.190@KRBTEST.COM (short username kudu)
I20260809 02:19:14.101315 7588 webserver.cc:460] Webserver started at http://127.15.75.190:36907/ using document root <none> and password file <none>
I20260809 02:19:14.101884 7588 fs_manager.cc:359] Metadata directory not provided
I20260809 02:19:14.101980 7588 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:19:14.105682 7588 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260809 02:19:14.107793 7614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:19:14.109086 7588 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 02:19:14.109226 7588 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/master-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/master-0/wal
uuid: "a708a9aa21ba4915b9ae3f1ff4de527c"
format_stamp: "Formatted at 2026-08-09 02:18:52 on dist-test-slave-s11t"
server_key: "5d58d775c4e31ddf3fd36fd7b0e4d8ad"
server_key_iv: "193ea38e7521fe51bb2e9302c4109c7f"
server_key_version: "encryptionkey@0"
I20260809 02:19:14.109798 7588 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:19:14.139307 7588 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:19:14.142168 7588 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:19:14.142378 7588 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:19:14.151597 7588 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.190:36755
I20260809 02:19:14.151904 7698 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.190:36755 every 8 connection(s)
I20260809 02:19:14.152729 7588 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
I20260809 02:19:14.157009 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 7588
I20260809 02:19:14.157320 7700 sys_catalog.cc:263] Verifying existing consensus state
I20260809 02:19:14.157539 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.129:40471
--local_ip_for_outbound_sockets=127.15.75.129
--tserver_master_addrs=127.15.75.190:36755
--webserver_port=38441
--webserver_interface=127.15.75.129
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab
--principal=kudu/127.15.75.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:45655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 02:19:14.159858 7700 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a708a9aa21ba4915b9ae3f1ff4de527c: Bootstrap starting.
I20260809 02:19:14.177353 7700 log.cc:824] T 00000000000000000000000000000000 P a708a9aa21ba4915b9ae3f1ff4de527c: Log is configured to *not* fsync() on all Append() calls
I20260809 02:19:14.186041 7700 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a708a9aa21ba4915b9ae3f1ff4de527c: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 02:19:14.186410 7700 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a708a9aa21ba4915b9ae3f1ff4de527c: Bootstrap complete.
I20260809 02:19:14.190491 7700 raft_consensus.cc:348] T 00000000000000000000000000000000 P a708a9aa21ba4915b9ae3f1ff4de527c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a708a9aa21ba4915b9ae3f1ff4de527c" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 36755 } }
I20260809 02:19:14.190852 7700 raft_consensus.cc:729] T 00000000000000000000000000000000 P a708a9aa21ba4915b9ae3f1ff4de527c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a708a9aa21ba4915b9ae3f1ff4de527c, State: Initialized, Role: FOLLOWER
I20260809 02:19:14.191434 7700 consensus_queue.cc:260] T 00000000000000000000000000000000 P a708a9aa21ba4915b9ae3f1ff4de527c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a708a9aa21ba4915b9ae3f1ff4de527c" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 36755 } }
I20260809 02:19:14.191553 7700 raft_consensus.cc:388] T 00000000000000000000000000000000 P a708a9aa21ba4915b9ae3f1ff4de527c [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:19:14.191642 7700 raft_consensus.cc:482] T 00000000000000000000000000000000 P a708a9aa21ba4915b9ae3f1ff4de527c [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:19:14.191751 7700 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a708a9aa21ba4915b9ae3f1ff4de527c [term 1 FOLLOWER]: Advancing to term 2
I20260809 02:19:14.193710 7700 raft_consensus.cc:504] T 00000000000000000000000000000000 P a708a9aa21ba4915b9ae3f1ff4de527c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a708a9aa21ba4915b9ae3f1ff4de527c" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 36755 } }
I20260809 02:19:14.194049 7700 leader_election.cc:302] T 00000000000000000000000000000000 P a708a9aa21ba4915b9ae3f1ff4de527c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a708a9aa21ba4915b9ae3f1ff4de527c; no voters:
I20260809 02:19:14.194375 7700 leader_election.cc:288] T 00000000000000000000000000000000 P a708a9aa21ba4915b9ae3f1ff4de527c [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 02:19:14.194463 7706 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a708a9aa21ba4915b9ae3f1ff4de527c [term 2 FOLLOWER]: Leader election won for term 2
I20260809 02:19:14.194662 7706 raft_consensus.cc:686] T 00000000000000000000000000000000 P a708a9aa21ba4915b9ae3f1ff4de527c [term 2 LEADER]: Becoming Leader. State: Replica: a708a9aa21ba4915b9ae3f1ff4de527c, State: Running, Role: LEADER
I20260809 02:19:14.194947 7706 consensus_queue.cc:237] T 00000000000000000000000000000000 P a708a9aa21ba4915b9ae3f1ff4de527c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a708a9aa21ba4915b9ae3f1ff4de527c" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 36755 } }
I20260809 02:19:14.195585 7700 sys_catalog.cc:564] T 00000000000000000000000000000000 P a708a9aa21ba4915b9ae3f1ff4de527c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:19:14.195850 7706 sys_catalog.cc:455] T 00000000000000000000000000000000 P a708a9aa21ba4915b9ae3f1ff4de527c [sys.catalog]: SysCatalogTable state changed. Reason: New leader a708a9aa21ba4915b9ae3f1ff4de527c. Latest consensus state: current_term: 2 leader_uuid: "a708a9aa21ba4915b9ae3f1ff4de527c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a708a9aa21ba4915b9ae3f1ff4de527c" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 36755 } } }
I20260809 02:19:14.195971 7706 sys_catalog.cc:458] T 00000000000000000000000000000000 P a708a9aa21ba4915b9ae3f1ff4de527c [sys.catalog]: This master's current role is: LEADER
I20260809 02:19:14.196329 7713 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:19:14.196434 7707 sys_catalog.cc:455] T 00000000000000000000000000000000 P a708a9aa21ba4915b9ae3f1ff4de527c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "a708a9aa21ba4915b9ae3f1ff4de527c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a708a9aa21ba4915b9ae3f1ff4de527c" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 36755 } } }
I20260809 02:19:14.196602 7707 sys_catalog.cc:458] T 00000000000000000000000000000000 P a708a9aa21ba4915b9ae3f1ff4de527c [sys.catalog]: This master's current role is: LEADER
I20260809 02:19:14.201144 7713 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:19:14.201814 7713 catalog_manager.cc:1259] Loaded cluster ID: 60c1e4383e294c28931f3fe91c6df02b
I20260809 02:19:14.201864 7713 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:19:14.203655 7713 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:19:14.204270 7713 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a708a9aa21ba4915b9ae3f1ff4de527c: Loaded TSK: 0
I20260809 02:19:14.222254 7713 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a708a9aa21ba4915b9ae3f1ff4de527c: Generated new TSK 1
I20260809 02:19:14.222493 7713 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 02:19:14.289402 7702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:19:14.289655 7702 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:19:14.289728 7702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:19:14.293692 7702 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:19:14.293763 7702 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:19:14.293846 7702 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.129
I20260809 02:19:14.299427 7702 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:45655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab
--principal=kudu/127.15.75.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.129:40471
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.75.129
--webserver_port=38441
--tserver_master_addrs=127.15.75.190:36755
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7702
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.129
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:19:14.300408 7702 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:19:14.301467 7702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:19:14.310994 7739 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:19:14.311036 7743 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:19:14.311017 7738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:19:14.311296 7702 server_base.cc:1034] running on GCE node
I20260809 02:19:14.311733 7702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:19:14.312382 7702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:19:14.313567 7702 hybrid_clock.cc:648] HybridClock initialized: now 1786241954313528 us; error 37 us; skew 500 ppm
Aug 09 02:19:14 dist-test-slave-s11t krb5kdc[6753](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241954, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:19:14.316335 7702 init.cc:377] Logged in from keytab as kudu/127.15.75.129@KRBTEST.COM (short username kudu)
I20260809 02:19:14.317294 7702 webserver.cc:460] Webserver started at http://127.15.75.129:38441/ using document root <none> and password file <none>
I20260809 02:19:14.317827 7702 fs_manager.cc:359] Metadata directory not provided
I20260809 02:19:14.317925 7702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:19:14.321662 7702 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260809 02:19:14.323846 7753 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:19:14.325145 7702 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 02:19:14.325274 7702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-0/wal
uuid: "8503d46e570848f0af6333917d9fee07"
format_stamp: "Formatted at 2026-08-09 02:18:53 on dist-test-slave-s11t"
server_key: "08b453b8585072939f4b0089a3953574"
server_key_iv: "2f4271788fd852c24df45b4202c41058"
server_key_version: "encryptionkey@0"
I20260809 02:19:14.325798 7702 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:19:14.378434 7702 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:19:14.381382 7702 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:19:14.381588 7702 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:19:14.382216 7702 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:19:14.383322 7761 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 02:19:14.387471 7702 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 02:19:14.387562 7702 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.000s
I20260809 02:19:14.387645 7702 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 02:19:14.389309 7702 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 02:19:14.389377 7702 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260809 02:19:14.389643 7761 tablet_bootstrap.cc:492] T 3be740b2f89a415f97a0ab7110c7b18f P 8503d46e570848f0af6333917d9fee07: Bootstrap starting.
I20260809 02:19:14.401176 7702 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.129:40471
I20260809 02:19:14.401188 7921 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.129:40471 every 8 connection(s)
I20260809 02:19:14.402388 7702 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
I20260809 02:19:14.405690 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 7702
I20260809 02:19:14.406198 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.130:45381
--local_ip_for_outbound_sockets=127.15.75.130
--tserver_master_addrs=127.15.75.190:36755
--webserver_port=43373
--webserver_interface=127.15.75.130
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab
--principal=kudu/127.15.75.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:45655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 02:19:14.410311 7761 log.cc:824] T 3be740b2f89a415f97a0ab7110c7b18f P 8503d46e570848f0af6333917d9fee07: Log is configured to *not* fsync() on all Append() calls
Aug 09 02:19:14 dist-test-slave-s11t krb5kdc[6753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241954, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.129@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:19:14.420117 7761 tablet_bootstrap.cc:492] T 3be740b2f89a415f97a0ab7110c7b18f P 8503d46e570848f0af6333917d9fee07: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 02:19:14.420348 7929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:14.404485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36755 (local address 127.15.75.129:44975)
0809 02:19:14.405202 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:14.405253 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 02:19:14.406379 (+ 1126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:19:14.407792 (+ 1413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:19:14.407804 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:19:14.408229 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:19:14.408750 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:14.408778 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:14.410644 (+ 1866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:14.410647 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:14.411794 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:14.411809 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:14.412870 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:14.412873 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:14.413053 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:14.413641 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:19:14.413696 (+ 55us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:19:14.415257 (+ 1561us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:19:14.417368 (+ 2111us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:19:14.417374 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:19:14.417386 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:19:14.417647 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:19:14.418000 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:19:14.418003 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:19:14.418005 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:19:14.418140 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:19:14.418699 (+ 559us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:19:14.418705 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:19:14.418950 (+ 245us) client_negotiation.cc:769] Sending connection context
0809 02:19:14.419126 (+ 176us) client_negotiation.cc:240] Negotiation successful
0809 02:19:14.419450 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":172,"threads_started":1}
I20260809 02:19:14.420626 7761 tablet_bootstrap.cc:492] T 3be740b2f89a415f97a0ab7110c7b18f P 8503d46e570848f0af6333917d9fee07: Bootstrap complete.
I20260809 02:19:14.420382 7928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:14.404510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.129:44975 (local address 127.15.75.190:36755)
0809 02:19:14.404910 (+ 400us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:14.404931 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:14.405389 (+ 458us) server_negotiation.cc:407] Connection header received
0809 02:19:14.406886 (+ 1497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:14.406909 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:14.407322 (+ 413us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:14.407696 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:14.408974 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:14.410492 (+ 1518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:14.411949 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:14.412430 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:14.412526 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:14.415459 (+ 2933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:19:14.415464 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:19:14.415476 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:19:14.415502 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:19:14.417243 (+ 1741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:14.417778 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:14.417783 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:14.417787 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:14.417902 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:14.418249 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:14.418253 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:14.418254 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:14.418582 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:19:14.418726 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:14.419260 (+ 534us) server_negotiation.cc:299] Negotiation successful
0809 02:19:14.419534 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":145,"threads_started":1}
I20260809 02:19:14.422194 7761 ts_tablet_manager.cc:1397] T 3be740b2f89a415f97a0ab7110c7b18f P 8503d46e570848f0af6333917d9fee07: Time spent bootstrapping tablet: real 0.033s user 0.025s sys 0.006s
I20260809 02:19:14.422901 7922 heartbeater.cc:347] Connected to a master server at 127.15.75.190:36755
I20260809 02:19:14.423127 7922 heartbeater.cc:464] Registering TS with master...
I20260809 02:19:14.423579 7922 heartbeater.cc:511] Master 127.15.75.190:36755 requested a full tablet report, sending...
I20260809 02:19:14.425057 7761 raft_consensus.cc:348] T 3be740b2f89a415f97a0ab7110c7b18f P 8503d46e570848f0af6333917d9fee07 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29341fbfcc86453da034b2c10e418b3e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45381 } } peers { permanent_uuid: "4ba2d99b0bd74c71b736bed8f281e453" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45603 } } peers { permanent_uuid: "8503d46e570848f0af6333917d9fee07" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 40471 } }
I20260809 02:19:14.425568 7761 raft_consensus.cc:729] T 3be740b2f89a415f97a0ab7110c7b18f P 8503d46e570848f0af6333917d9fee07 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8503d46e570848f0af6333917d9fee07, State: Initialized, Role: FOLLOWER
I20260809 02:19:14.425963 7639 ts_manager.cc:194] Registered new tserver with Master: 8503d46e570848f0af6333917d9fee07 (127.15.75.129:40471)
I20260809 02:19:14.426038 7761 consensus_queue.cc:260] T 3be740b2f89a415f97a0ab7110c7b18f P 8503d46e570848f0af6333917d9fee07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29341fbfcc86453da034b2c10e418b3e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45381 } } peers { permanent_uuid: "4ba2d99b0bd74c71b736bed8f281e453" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45603 } } peers { permanent_uuid: "8503d46e570848f0af6333917d9fee07" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 40471 } }
I20260809 02:19:14.427042 7761 ts_tablet_manager.cc:1428] T 3be740b2f89a415f97a0ab7110c7b18f P 8503d46e570848f0af6333917d9fee07: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260809 02:19:14.429738 7639 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.75.129@KRBTEST.COM'} at 127.15.75.129:44975
I20260809 02:19:14.432430 7922 heartbeater.cc:503] Master 127.15.75.190:36755 was elected leader, sending a full tablet report...
I20260809 02:19:14.435844 7934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:14.428780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.129:40471 (local address 127.0.0.1:51766)
0809 02:19:14.430669 (+ 1889us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:14.430702 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 02:19:14.430842 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:19:14.431704 (+ 862us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:19:14.431711 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:19:14.431740 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:19:14.431872 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:14.431878 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:14.433299 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:14.433302 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:14.434497 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:14.434508 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:14.435471 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:14.435474 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:14.435552 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:14.435580 (+ 28us) client_negotiation.cc:769] Sending connection context
0809 02:19:14.435660 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 02:19:14.435743 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1735,"thread_start_us":139,"threads_started":1}
I20260809 02:19:14.435887 7935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:14.428856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51766 (local address 127.15.75.129:40471)
0809 02:19:14.429155 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:14.429167 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:14.430984 (+ 1817us) server_negotiation.cc:407] Connection header received
0809 02:19:14.431050 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:14.431055 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:14.431530 (+ 475us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:14.431684 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:19:14.432030 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:14.433183 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:14.434619 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:14.435358 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:14.435474 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:14.435600 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:14.435685 (+ 85us) server_negotiation.cc:299] Negotiation successful
0809 02:19:14.435768 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":118,"threads_started":1}
I20260809 02:19:14.437042 7830 tablet_service.cc:1514] Processing DeleteTablet for tablet 3be740b2f89a415f97a0ab7110c7b18f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:18:53 UTC) from {username='kudu', principal='kudu/127.15.75.190@KRBTEST.COM'} at 127.0.0.1:51766
I20260809 02:19:14.437906 7937 tablet_replica.cc:330] stopping tablet replica
I20260809 02:19:14.438172 7937 raft_consensus.cc:2229] T 3be740b2f89a415f97a0ab7110c7b18f P 8503d46e570848f0af6333917d9fee07 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:19:14.438400 7937 raft_consensus.cc:2258] T 3be740b2f89a415f97a0ab7110c7b18f P 8503d46e570848f0af6333917d9fee07 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:19:14.439252 7937 ts_tablet_manager.cc:1905] T 3be740b2f89a415f97a0ab7110c7b18f P 8503d46e570848f0af6333917d9fee07: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:19:14.442596 7937 ts_tablet_manager.cc:1918] T 3be740b2f89a415f97a0ab7110c7b18f P 8503d46e570848f0af6333917d9fee07: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:19:14.442718 7937 log.cc:1196] T 3be740b2f89a415f97a0ab7110c7b18f P 8503d46e570848f0af6333917d9fee07: Deleting WAL directory at /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-0/wal/wals/3be740b2f89a415f97a0ab7110c7b18f
I20260809 02:19:14.443007 7937 ts_tablet_manager.cc:1939] T 3be740b2f89a415f97a0ab7110c7b18f P 8503d46e570848f0af6333917d9fee07: Deleting consensus metadata
I20260809 02:19:14.444511 7619 catalog_manager.cc:4841] TS 8503d46e570848f0af6333917d9fee07 (127.15.75.129:40471): tablet 3be740b2f89a415f97a0ab7110c7b18f (table test-table [id=939fbd64d4634ff6a99314e90670f7c7]) successfully deleted
W20260809 02:19:14.525547 7931 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:19:14.525772 7931 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:19:14.525813 7931 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:19:14.530082 7931 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:19:14.530131 7931 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:19:14.530185 7931 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.130
I20260809 02:19:14.535791 7931 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:45655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab
--principal=kudu/127.15.75.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.130:45381
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.75.130
--webserver_port=43373
--tserver_master_addrs=127.15.75.190:36755
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7931
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.130
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:19:14.536705 7931 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:19:14.537624 7931 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:19:14.546909 7947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:19:14.546916 7948 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:19:14.547034 7931 server_base.cc:1034] running on GCE node
W20260809 02:19:14.546916 7950 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:19:14.547555 7931 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:19:14.548174 7931 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:19:14.549353 7931 hybrid_clock.cc:648] HybridClock initialized: now 1786241954549338 us; error 27 us; skew 500 ppm
Aug 09 02:19:14 dist-test-slave-s11t krb5kdc[6753](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241954, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:19:14.551978 7931 init.cc:377] Logged in from keytab as kudu/127.15.75.130@KRBTEST.COM (short username kudu)
I20260809 02:19:14.552851 7931 webserver.cc:460] Webserver started at http://127.15.75.130:43373/ using document root <none> and password file <none>
I20260809 02:19:14.553400 7931 fs_manager.cc:359] Metadata directory not provided
I20260809 02:19:14.553503 7931 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:19:14.557674 7931 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 02:19:14.559666 7960 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:19:14.560873 7931 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 02:19:14.560997 7931 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-1/wal
uuid: "29341fbfcc86453da034b2c10e418b3e"
format_stamp: "Formatted at 2026-08-09 02:18:53 on dist-test-slave-s11t"
server_key: "2f512ec72d10f1d90b5bfcd88748de8d"
server_key_iv: "b6d6f1df53f3364551c64d3ec5f66f4e"
server_key_version: "encryptionkey@0"
I20260809 02:19:14.561533 7931 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:19:14.578078 7931 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:19:14.580631 7931 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:19:14.580842 7931 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:19:14.581409 7931 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:19:14.582195 7931 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:19:14.582265 7931 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:19:14.582334 7931 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:19:14.582372 7931 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:19:14.591374 7931 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.130:45381
I20260809 02:19:14.591430 8130 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.130:45381 every 8 connection(s)
I20260809 02:19:14.592326 7931 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
I20260809 02:19:14.595175 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 7931
I20260809 02:19:14.595625 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.131:45603
--local_ip_for_outbound_sockets=127.15.75.131
--tserver_master_addrs=127.15.75.190:36755
--webserver_port=36691
--webserver_interface=127.15.75.131
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab
--principal=kudu/127.15.75.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:45655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 02:19:14 dist-test-slave-s11t krb5kdc[6753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241954, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.130@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:19:14.605798 7928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:14.594113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.130:43871 (local address 127.15.75.190:36755)
0809 02:19:14.594266 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:14.594271 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:14.595096 (+ 825us) server_negotiation.cc:407] Connection header received
0809 02:19:14.596608 (+ 1512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:14.596611 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:14.596676 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:14.596778 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:14.597922 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:14.598456 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:14.599565 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:14.599809 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:14.599927 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:14.602328 (+ 2401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:19:14.602332 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:19:14.602334 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:19:14.602358 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:19:14.603785 (+ 1427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:14.604304 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:14.604308 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:14.604310 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:14.604390 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:14.604730 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:14.604733 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:14.604735 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:14.604921 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:19:14.605024 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:14.605491 (+ 467us) server_negotiation.cc:299] Negotiation successful
0809 02:19:14.605625 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 02:19:14.606516 8133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:14.594503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36755 (local address 127.15.75.130:43871)
0809 02:19:14.594942 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:14.594982 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 02:19:14.596404 (+ 1422us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:19:14.596940 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:19:14.596947 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:19:14.597321 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:19:14.597768 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:14.597780 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:14.598576 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:14.598580 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:14.599439 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:14.599448 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:14.599956 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:14.599961 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:14.600127 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:14.600789 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:19:14.600805 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:19:14.602166 (+ 1361us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:19:14.603900 (+ 1734us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:19:14.603905 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:19:14.603920 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:19:14.604192 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:19:14.604479 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:19:14.604483 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:19:14.604485 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:19:14.604617 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:19:14.605036 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:19:14.605044 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:19:14.605266 (+ 222us) client_negotiation.cc:769] Sending connection context
0809 02:19:14.605450 (+ 184us) client_negotiation.cc:240] Negotiation successful
0809 02:19:14.605745 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":129,"threads_started":1}
I20260809 02:19:14.607647 8131 heartbeater.cc:347] Connected to a master server at 127.15.75.190:36755
I20260809 02:19:14.607877 8131 heartbeater.cc:464] Registering TS with master...
I20260809 02:19:14.608338 8131 heartbeater.cc:511] Master 127.15.75.190:36755 requested a full tablet report, sending...
I20260809 02:19:14.609408 7639 ts_manager.cc:194] Registered new tserver with Master: 29341fbfcc86453da034b2c10e418b3e (127.15.75.130:45381)
I20260809 02:19:14.610150 7639 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.75.130@KRBTEST.COM'} at 127.15.75.130:43871
W20260809 02:19:14.709240 8134 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:19:14.709478 8134 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:19:14.709519 8134 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:19:14.713548 8134 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:19:14.713656 8134 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:19:14.713728 8134 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.131
I20260809 02:19:14.718880 8134 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:45655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab
--principal=kudu/127.15.75.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.131:45603
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.75.131
--webserver_port=36691
--tserver_master_addrs=127.15.75.190:36755
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8134
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.131
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:19:14.719785 8134 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:19:14.720726 8134 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:19:14.730384 8142 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:19:14.730365 8140 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:19:14.730360 8144 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:19:14.730607 8134 server_base.cc:1034] running on GCE node
I20260809 02:19:14.731045 8134 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:19:14.731628 8134 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:19:14.732800 8134 hybrid_clock.cc:648] HybridClock initialized: now 1786241954732776 us; error 42 us; skew 500 ppm
Aug 09 02:19:14 dist-test-slave-s11t krb5kdc[6753](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241954, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:19:14.735437 8134 init.cc:377] Logged in from keytab as kudu/127.15.75.131@KRBTEST.COM (short username kudu)
I20260809 02:19:14.736224 8134 webserver.cc:460] Webserver started at http://127.15.75.131:36691/ using document root <none> and password file <none>
I20260809 02:19:14.736807 8134 fs_manager.cc:359] Metadata directory not provided
I20260809 02:19:14.736917 8134 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:19:14.740473 8134 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260809 02:19:14.742506 8163 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:19:14.743789 8134 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 02:19:14.743927 8134 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-2/wal
uuid: "4ba2d99b0bd74c71b736bed8f281e453"
format_stamp: "Formatted at 2026-08-09 02:18:53 on dist-test-slave-s11t"
server_key: "2d5bef98351dd6812a39e5b7901a1654"
server_key_iv: "9baa0bb5e138d4a88823d87de26ce886"
server_key_version: "encryptionkey@0"
I20260809 02:19:14.744439 8134 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:19:14.767692 8134 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:19:14.770455 8134 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:19:14.770653 8134 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:19:14.771245 8134 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:19:14.772048 8134 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:19:14.772117 8134 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:19:14.772185 8134 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:19:14.772225 8134 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:19:14.781056 8134 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.131:45603
I20260809 02:19:14.781098 8343 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.131:45603 every 8 connection(s)
I20260809 02:19:14.782021 8134 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
I20260809 02:19:14.783087 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 8134
I20260809 02:19:14.792742 7928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:14.783796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56942 (local address 127.15.75.190:36755)
0809 02:19:14.783942 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:14.783948 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:14.784163 (+ 215us) server_negotiation.cc:407] Connection header received
0809 02:19:14.784307 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:14.784310 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:14.784364 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:14.784455 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:14.785295 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:14.785958 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:14.787150 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:14.787361 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:14.787443 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:14.788898 (+ 1455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:19:14.788901 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:19:14.788903 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:19:14.788924 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:19:14.790408 (+ 1484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:14.791020 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:14.791026 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:14.791027 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:14.791124 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:14.791476 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:14.791481 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:14.791485 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:14.791700 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:19:14.791955 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:14.792374 (+ 419us) server_negotiation.cc:299] Negotiation successful
0809 02:19:14.792491 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
Aug 09 02:19:14 dist-test-slave-s11t krb5kdc[6753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241954, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.131@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:19:14.797434 8347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:14.784083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.131:37889 (local address 127.15.75.190:36755)
0809 02:19:14.784306 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:14.784310 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:14.785487 (+ 1177us) server_negotiation.cc:407] Connection header received
0809 02:19:14.786474 (+ 987us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:14.786476 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:14.786513 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:14.786575 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:14.788034 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:14.788544 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:14.789800 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:14.790814 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:14.790919 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:14.794056 (+ 3137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:19:14.794060 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:19:14.794062 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:19:14.794090 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:19:14.795612 (+ 1522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:14.796070 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:14.796072 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:14.796073 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:14.796147 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:14.796480 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:14.796482 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:14.796484 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:14.796665 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:19:14.796794 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:14.797223 (+ 429us) server_negotiation.cc:299] Negotiation successful
0809 02:19:14.797312 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":62,"threads_started":1}
I20260809 02:19:14.798197 8348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:14.784374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36755 (local address 127.15.75.131:37889)
0809 02:19:14.785353 (+ 979us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:14.785400 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 02:19:14.786260 (+ 860us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:19:14.786753 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:19:14.786761 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:19:14.787253 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:19:14.787875 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:14.787892 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:14.788671 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:14.788675 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:14.789649 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:14.789659 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:14.790993 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:14.791001 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:14.791274 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:14.791990 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:19:14.792013 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:19:14.793883 (+ 1870us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:19:14.795712 (+ 1829us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:19:14.795716 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:19:14.795727 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:19:14.795977 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:19:14.796220 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:19:14.796223 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:19:14.796224 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:19:14.796347 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:19:14.796800 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:19:14.796805 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:19:14.797017 (+ 212us) client_negotiation.cc:769] Sending connection context
0809 02:19:14.797165 (+ 148us) client_negotiation.cc:240] Negotiation successful
0809 02:19:14.797404 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":754,"thread_start_us":590,"threads_started":1}
I20260809 02:19:14.799218 8344 heartbeater.cc:347] Connected to a master server at 127.15.75.190:36755
I20260809 02:19:14.799453 8344 heartbeater.cc:464] Registering TS with master...
I20260809 02:19:14.799916 8344 heartbeater.cc:511] Master 127.15.75.190:36755 requested a full tablet report, sending...
I20260809 02:19:14.800894 7639 ts_manager.cc:194] Registered new tserver with Master: 4ba2d99b0bd74c71b736bed8f281e453 (127.15.75.131:45603)
I20260809 02:19:14.801436 7639 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.75.131@KRBTEST.COM'} at 127.15.75.131:37889
I20260809 02:19:14.806608 15662 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 02:19:14.810066 8347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:14.807150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56954 (local address 127.15.75.190:36755)
0809 02:19:14.807249 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:14.807252 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:14.807411 (+ 159us) server_negotiation.cc:407] Connection header received
0809 02:19:14.807528 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:14.807530 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:14.807609 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:14.807686 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:19:14.807874 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:14.808333 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:14.809177 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:14.809384 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:14.809454 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:14.809690 (+ 236us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:19:14.809800 (+ 110us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0809 02:19:14.809989 (+ 189us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:56954: token expired
Metrics: {"server-negotiator.queue_time_us":37}
W20260809 02:19:14.810141 8347 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:56954: token expired
W20260809 02:19:14.810483 7526 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.15.75.190:36755)
I20260809 02:19:14.817454 8347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:14.810835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56970 (local address 127.15.75.190:36755)
0809 02:19:14.810958 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:14.810963 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:14.810976 (+ 13us) server_negotiation.cc:407] Connection header received
0809 02:19:14.811085 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:14.811088 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:14.811128 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:14.811203 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:14.811768 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:14.812221 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:14.813120 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:14.813321 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:14.813428 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:14.814660 (+ 1232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:19:14.814664 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:19:14.814667 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:19:14.814695 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:19:14.815812 (+ 1117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:14.816222 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:14.816226 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:14.816228 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:14.816305 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:14.816566 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:14.816571 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:14.816573 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:14.816839 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:19:14.816972 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:14.817195 (+ 223us) server_negotiation.cc:299] Negotiation successful
0809 02:19:14.817307 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 02:19:14.819556 7639 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56970:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:19:14.822757 7639 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:19:14.830696 7830 tablet_service.cc:1467] Processing CreateTablet for tablet 1a3b6ba630cc434d8898d43027e70e07 (DEFAULT_TABLE table=test-table [id=1a2080538d2e4967b9b51d849ea21591]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:19:14.831360 7830 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a3b6ba630cc434d8898d43027e70e07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:19:14.834605 7934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:14.829098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.130:45381 (local address 127.0.0.1:34374)
0809 02:19:14.829225 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:14.829240 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:19:14.829316 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:19:14.830362 (+ 1046us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:19:14.830366 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:19:14.830384 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:19:14.830483 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:14.830489 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:14.831756 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:14.831761 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:14.832962 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:14.832973 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:14.834153 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:14.834158 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:14.834263 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:14.834305 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 02:19:14.834377 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 02:19:14.834455 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45,"mutex_wait_us":4}
I20260809 02:19:14.834613 7761 tablet_bootstrap.cc:492] T 1a3b6ba630cc434d8898d43027e70e07 P 8503d46e570848f0af6333917d9fee07: Bootstrap starting.
I20260809 02:19:14.834491 8359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:14.829651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.131:45603 (local address 127.0.0.1:39560)
0809 02:19:14.829905 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:14.829919 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:19:14.830008 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:19:14.830707 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:19:14.830711 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:19:14.830728 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:19:14.830810 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:14.830815 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:14.832043 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:14.832046 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:14.833188 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:14.833195 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:14.834085 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:14.834089 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:14.834187 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:14.834208 (+ 21us) client_negotiation.cc:769] Sending connection context
0809 02:19:14.834292 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 02:19:14.834362 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"mutex_wait_us":41,"thread_start_us":97,"threads_started":1}
I20260809 02:19:14.835022 8356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:14.829294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34374 (local address 127.15.75.130:45381)
0809 02:19:14.829717 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:14.829733 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:14.829760 (+ 27us) server_negotiation.cc:407] Connection header received
0809 02:19:14.829840 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:14.829845 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:14.830178 (+ 333us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:14.830394 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:19:14.830567 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:14.831653 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:14.833087 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:14.834005 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:14.834570 (+ 565us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:14.834713 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:14.834767 (+ 54us) server_negotiation.cc:299] Negotiation successful
0809 02:19:14.834854 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":139,"threads_started":1}
I20260809 02:19:14.835450 7761 tablet_bootstrap.cc:654] T 1a3b6ba630cc434d8898d43027e70e07 P 8503d46e570848f0af6333917d9fee07: Neither blocks nor log segments found. Creating new log.
I20260809 02:19:14.836452 7761 tablet_bootstrap.cc:492] T 1a3b6ba630cc434d8898d43027e70e07 P 8503d46e570848f0af6333917d9fee07: No bootstrap required, opened a new log
I20260809 02:19:14.836524 7761 ts_tablet_manager.cc:1397] T 1a3b6ba630cc434d8898d43027e70e07 P 8503d46e570848f0af6333917d9fee07: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 02:19:14.836761 8360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:14.829792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39560 (local address 127.15.75.131:45603)
0809 02:19:14.830270 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:14.830281 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:14.830304 (+ 23us) server_negotiation.cc:407] Connection header received
0809 02:19:14.830386 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:14.830391 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:14.830567 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:14.830726 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:19:14.830916 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:14.831924 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:14.833292 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:14.833963 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:14.834080 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:14.834223 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:14.834322 (+ 99us) server_negotiation.cc:299] Negotiation successful
0809 02:19:14.834416 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":114,"threads_started":1}
I20260809 02:19:14.836993 7761 raft_consensus.cc:348] T 1a3b6ba630cc434d8898d43027e70e07 P 8503d46e570848f0af6333917d9fee07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29341fbfcc86453da034b2c10e418b3e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45381 } } peers { permanent_uuid: "4ba2d99b0bd74c71b736bed8f281e453" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45603 } } peers { permanent_uuid: "8503d46e570848f0af6333917d9fee07" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 40471 } }
I20260809 02:19:14.837108 7761 raft_consensus.cc:374] T 1a3b6ba630cc434d8898d43027e70e07 P 8503d46e570848f0af6333917d9fee07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:19:14.837157 7761 raft_consensus.cc:729] T 1a3b6ba630cc434d8898d43027e70e07 P 8503d46e570848f0af6333917d9fee07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8503d46e570848f0af6333917d9fee07, State: Initialized, Role: FOLLOWER
I20260809 02:19:14.837285 7761 consensus_queue.cc:260] T 1a3b6ba630cc434d8898d43027e70e07 P 8503d46e570848f0af6333917d9fee07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29341fbfcc86453da034b2c10e418b3e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45381 } } peers { permanent_uuid: "4ba2d99b0bd74c71b736bed8f281e453" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45603 } } peers { permanent_uuid: "8503d46e570848f0af6333917d9fee07" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 40471 } }
I20260809 02:19:14.837569 7761 ts_tablet_manager.cc:1428] T 1a3b6ba630cc434d8898d43027e70e07 P 8503d46e570848f0af6333917d9fee07: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 02:19:14.837941 8037 tablet_service.cc:1467] Processing CreateTablet for tablet 1a3b6ba630cc434d8898d43027e70e07 (DEFAULT_TABLE table=test-table [id=1a2080538d2e4967b9b51d849ea21591]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:19:14.839075 8037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a3b6ba630cc434d8898d43027e70e07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:19:14.839126 8236 tablet_service.cc:1467] Processing CreateTablet for tablet 1a3b6ba630cc434d8898d43027e70e07 (DEFAULT_TABLE table=test-table [id=1a2080538d2e4967b9b51d849ea21591]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:19:14.839931 8236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a3b6ba630cc434d8898d43027e70e07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:19:14.845927 8363 tablet_bootstrap.cc:492] T 1a3b6ba630cc434d8898d43027e70e07 P 4ba2d99b0bd74c71b736bed8f281e453: Bootstrap starting.
I20260809 02:19:14.845927 8362 tablet_bootstrap.cc:492] T 1a3b6ba630cc434d8898d43027e70e07 P 29341fbfcc86453da034b2c10e418b3e: Bootstrap starting.
I20260809 02:19:14.847950 8363 tablet_bootstrap.cc:654] T 1a3b6ba630cc434d8898d43027e70e07 P 4ba2d99b0bd74c71b736bed8f281e453: Neither blocks nor log segments found. Creating new log.
I20260809 02:19:14.848520 8362 tablet_bootstrap.cc:654] T 1a3b6ba630cc434d8898d43027e70e07 P 29341fbfcc86453da034b2c10e418b3e: Neither blocks nor log segments found. Creating new log.
I20260809 02:19:14.849424 8362 log.cc:824] T 1a3b6ba630cc434d8898d43027e70e07 P 29341fbfcc86453da034b2c10e418b3e: Log is configured to *not* fsync() on all Append() calls
I20260809 02:19:14.849532 8363 log.cc:824] T 1a3b6ba630cc434d8898d43027e70e07 P 4ba2d99b0bd74c71b736bed8f281e453: Log is configured to *not* fsync() on all Append() calls
I20260809 02:19:14.851214 8362 tablet_bootstrap.cc:492] T 1a3b6ba630cc434d8898d43027e70e07 P 29341fbfcc86453da034b2c10e418b3e: No bootstrap required, opened a new log
I20260809 02:19:14.851423 8362 ts_tablet_manager.cc:1397] T 1a3b6ba630cc434d8898d43027e70e07 P 29341fbfcc86453da034b2c10e418b3e: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 02:19:14.853987 8363 tablet_bootstrap.cc:492] T 1a3b6ba630cc434d8898d43027e70e07 P 4ba2d99b0bd74c71b736bed8f281e453: No bootstrap required, opened a new log
I20260809 02:19:14.854178 8363 ts_tablet_manager.cc:1397] T 1a3b6ba630cc434d8898d43027e70e07 P 4ba2d99b0bd74c71b736bed8f281e453: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.003s
I20260809 02:19:14.854853 8362 raft_consensus.cc:348] T 1a3b6ba630cc434d8898d43027e70e07 P 29341fbfcc86453da034b2c10e418b3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29341fbfcc86453da034b2c10e418b3e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45381 } } peers { permanent_uuid: "4ba2d99b0bd74c71b736bed8f281e453" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45603 } } peers { permanent_uuid: "8503d46e570848f0af6333917d9fee07" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 40471 } }
I20260809 02:19:14.855131 8362 raft_consensus.cc:374] T 1a3b6ba630cc434d8898d43027e70e07 P 29341fbfcc86453da034b2c10e418b3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:19:14.855192 8362 raft_consensus.cc:729] T 1a3b6ba630cc434d8898d43027e70e07 P 29341fbfcc86453da034b2c10e418b3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29341fbfcc86453da034b2c10e418b3e, State: Initialized, Role: FOLLOWER
I20260809 02:19:14.855644 8362 consensus_queue.cc:260] T 1a3b6ba630cc434d8898d43027e70e07 P 29341fbfcc86453da034b2c10e418b3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29341fbfcc86453da034b2c10e418b3e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45381 } } peers { permanent_uuid: "4ba2d99b0bd74c71b736bed8f281e453" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45603 } } peers { permanent_uuid: "8503d46e570848f0af6333917d9fee07" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 40471 } }
I20260809 02:19:14.856288 8131 heartbeater.cc:503] Master 127.15.75.190:36755 was elected leader, sending a full tablet report...
I20260809 02:19:14.856647 8362 ts_tablet_manager.cc:1428] T 1a3b6ba630cc434d8898d43027e70e07 P 29341fbfcc86453da034b2c10e418b3e: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260809 02:19:14.857613 8363 raft_consensus.cc:348] T 1a3b6ba630cc434d8898d43027e70e07 P 4ba2d99b0bd74c71b736bed8f281e453 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29341fbfcc86453da034b2c10e418b3e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45381 } } peers { permanent_uuid: "4ba2d99b0bd74c71b736bed8f281e453" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45603 } } peers { permanent_uuid: "8503d46e570848f0af6333917d9fee07" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 40471 } }
I20260809 02:19:14.857803 8363 raft_consensus.cc:374] T 1a3b6ba630cc434d8898d43027e70e07 P 4ba2d99b0bd74c71b736bed8f281e453 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:19:14.857864 8363 raft_consensus.cc:729] T 1a3b6ba630cc434d8898d43027e70e07 P 4ba2d99b0bd74c71b736bed8f281e453 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ba2d99b0bd74c71b736bed8f281e453, State: Initialized, Role: FOLLOWER
I20260809 02:19:14.858274 8363 consensus_queue.cc:260] T 1a3b6ba630cc434d8898d43027e70e07 P 4ba2d99b0bd74c71b736bed8f281e453 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29341fbfcc86453da034b2c10e418b3e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45381 } } peers { permanent_uuid: "4ba2d99b0bd74c71b736bed8f281e453" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45603 } } peers { permanent_uuid: "8503d46e570848f0af6333917d9fee07" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 40471 } }
I20260809 02:19:14.859149 8344 heartbeater.cc:503] Master 127.15.75.190:36755 was elected leader, sending a full tablet report...
I20260809 02:19:14.859212 8363 ts_tablet_manager.cc:1428] T 1a3b6ba630cc434d8898d43027e70e07 P 4ba2d99b0bd74c71b736bed8f281e453: Time spent starting tablet: real 0.005s user 0.004s sys 0.001s
I20260809 02:19:14.879195 8367 raft_consensus.cc:482] T 1a3b6ba630cc434d8898d43027e70e07 P 29341fbfcc86453da034b2c10e418b3e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:19:14.879421 8367 raft_consensus.cc:504] T 1a3b6ba630cc434d8898d43027e70e07 P 29341fbfcc86453da034b2c10e418b3e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29341fbfcc86453da034b2c10e418b3e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45381 } } peers { permanent_uuid: "4ba2d99b0bd74c71b736bed8f281e453" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45603 } } peers { permanent_uuid: "8503d46e570848f0af6333917d9fee07" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 40471 } }
I20260809 02:19:14.880530 8367 leader_election.cc:288] T 1a3b6ba630cc434d8898d43027e70e07 P 29341fbfcc86453da034b2c10e418b3e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ba2d99b0bd74c71b736bed8f281e453 (127.15.75.131:45603), 8503d46e570848f0af6333917d9fee07 (127.15.75.129:40471)
I20260809 02:19:14.884274 8133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:14.880776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.131:45603 (local address 127.15.75.130:50621)
0809 02:19:14.880893 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:14.880911 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 02:19:14.881021 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:19:14.881344 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:19:14.881350 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:19:14.881370 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:19:14.881476 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:14.881481 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:14.882196 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:14.882201 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:14.882982 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:14.882991 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:14.883845 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:14.883849 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:14.883969 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:14.883997 (+ 28us) client_negotiation.cc:769] Sending connection context
0809 02:19:14.884069 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 02:19:14.884149 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260809 02:19:14.884310 8360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:14.880849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.130:50621 (local address 127.15.75.131:45603)
0809 02:19:14.881009 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:14.881014 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:14.881033 (+ 19us) server_negotiation.cc:407] Connection header received
0809 02:19:14.881117 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:14.881120 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:14.881177 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:14.881261 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:19:14.881589 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:14.882087 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:14.883113 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:14.883729 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:14.883851 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:14.883903 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:14.884135 (+ 232us) server_negotiation.cc:299] Negotiation successful
0809 02:19:14.884207 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 02:19:14.885007 8268 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a3b6ba630cc434d8898d43027e70e07" candidate_uuid: "29341fbfcc86453da034b2c10e418b3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ba2d99b0bd74c71b736bed8f281e453" is_pre_election: true
I20260809 02:19:14.885305 8268 raft_consensus.cc:2454] T 1a3b6ba630cc434d8898d43027e70e07 P 4ba2d99b0bd74c71b736bed8f281e453 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29341fbfcc86453da034b2c10e418b3e in term 0.
I20260809 02:19:14.885368 8369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:14.881055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.129:40471 (local address 127.15.75.130:59213)
0809 02:19:14.881426 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:14.881436 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 02:19:14.881546 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:19:14.881797 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:19:14.881800 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:19:14.881815 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:19:14.881941 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:14.881947 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:14.882662 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:14.882667 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:14.883446 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:14.883454 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:14.884938 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:14.884942 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:14.885041 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:14.885083 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 02:19:14.885170 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 02:19:14.885239 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":122,"threads_started":1}
I20260809 02:19:14.885367 7935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:14.881182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.130:59213 (local address 127.15.75.129:40471)
0809 02:19:14.881315 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:14.881319 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:14.881515 (+ 196us) server_negotiation.cc:407] Connection header received
0809 02:19:14.881655 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:14.881658 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:14.881708 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:14.881784 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:19:14.882052 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:14.882550 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:14.883573 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:14.884791 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:14.884929 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:14.884980 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:14.885187 (+ 207us) server_negotiation.cc:299] Negotiation successful
0809 02:19:14.885263 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 02:19:14.885819 7963 leader_election.cc:302] T 1a3b6ba630cc434d8898d43027e70e07 P 29341fbfcc86453da034b2c10e418b3e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29341fbfcc86453da034b2c10e418b3e, 4ba2d99b0bd74c71b736bed8f281e453; no voters:
I20260809 02:19:14.886098 8367 raft_consensus.cc:2781] T 1a3b6ba630cc434d8898d43027e70e07 P 29341fbfcc86453da034b2c10e418b3e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:19:14.886083 7853 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a3b6ba630cc434d8898d43027e70e07" candidate_uuid: "29341fbfcc86453da034b2c10e418b3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8503d46e570848f0af6333917d9fee07" is_pre_election: true
I20260809 02:19:14.886195 8367 raft_consensus.cc:482] T 1a3b6ba630cc434d8898d43027e70e07 P 29341fbfcc86453da034b2c10e418b3e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:19:14.886245 8367 raft_consensus.cc:3037] T 1a3b6ba630cc434d8898d43027e70e07 P 29341fbfcc86453da034b2c10e418b3e [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:19:14.886373 7853 raft_consensus.cc:2454] T 1a3b6ba630cc434d8898d43027e70e07 P 8503d46e570848f0af6333917d9fee07 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29341fbfcc86453da034b2c10e418b3e in term 0.
I20260809 02:19:14.887360 8367 raft_consensus.cc:504] T 1a3b6ba630cc434d8898d43027e70e07 P 29341fbfcc86453da034b2c10e418b3e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29341fbfcc86453da034b2c10e418b3e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45381 } } peers { permanent_uuid: "4ba2d99b0bd74c71b736bed8f281e453" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45603 } } peers { permanent_uuid: "8503d46e570848f0af6333917d9fee07" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 40471 } }
I20260809 02:19:14.887645 8367 leader_election.cc:288] T 1a3b6ba630cc434d8898d43027e70e07 P 29341fbfcc86453da034b2c10e418b3e [CANDIDATE]: Term 1 election: Requested vote from peers 4ba2d99b0bd74c71b736bed8f281e453 (127.15.75.131:45603), 8503d46e570848f0af6333917d9fee07 (127.15.75.129:40471)
I20260809 02:19:14.887990 8268 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a3b6ba630cc434d8898d43027e70e07" candidate_uuid: "29341fbfcc86453da034b2c10e418b3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ba2d99b0bd74c71b736bed8f281e453"
I20260809 02:19:14.888108 8268 raft_consensus.cc:3037] T 1a3b6ba630cc434d8898d43027e70e07 P 4ba2d99b0bd74c71b736bed8f281e453 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:19:14.888100 7853 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a3b6ba630cc434d8898d43027e70e07" candidate_uuid: "29341fbfcc86453da034b2c10e418b3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8503d46e570848f0af6333917d9fee07"
I20260809 02:19:14.888227 7853 raft_consensus.cc:3037] T 1a3b6ba630cc434d8898d43027e70e07 P 8503d46e570848f0af6333917d9fee07 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:19:14.889132 8268 raft_consensus.cc:2454] T 1a3b6ba630cc434d8898d43027e70e07 P 4ba2d99b0bd74c71b736bed8f281e453 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29341fbfcc86453da034b2c10e418b3e in term 1.
I20260809 02:19:14.889380 7853 raft_consensus.cc:2454] T 1a3b6ba630cc434d8898d43027e70e07 P 8503d46e570848f0af6333917d9fee07 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29341fbfcc86453da034b2c10e418b3e in term 1.
I20260809 02:19:14.889429 7963 leader_election.cc:302] T 1a3b6ba630cc434d8898d43027e70e07 P 29341fbfcc86453da034b2c10e418b3e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29341fbfcc86453da034b2c10e418b3e, 4ba2d99b0bd74c71b736bed8f281e453; no voters:
I20260809 02:19:14.889596 8367 raft_consensus.cc:2781] T 1a3b6ba630cc434d8898d43027e70e07 P 29341fbfcc86453da034b2c10e418b3e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:19:14.889824 8367 raft_consensus.cc:686] T 1a3b6ba630cc434d8898d43027e70e07 P 29341fbfcc86453da034b2c10e418b3e [term 1 LEADER]: Becoming Leader. State: Replica: 29341fbfcc86453da034b2c10e418b3e, State: Running, Role: LEADER
I20260809 02:19:14.890151 8367 consensus_queue.cc:237] T 1a3b6ba630cc434d8898d43027e70e07 P 29341fbfcc86453da034b2c10e418b3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29341fbfcc86453da034b2c10e418b3e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45381 } } peers { permanent_uuid: "4ba2d99b0bd74c71b736bed8f281e453" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45603 } } peers { permanent_uuid: "8503d46e570848f0af6333917d9fee07" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 40471 } }
I20260809 02:19:14.893133 7639 catalog_manager.cc:5495] T 1a3b6ba630cc434d8898d43027e70e07 P 29341fbfcc86453da034b2c10e418b3e reported cstate change: term changed from 0 to 1, leader changed from <none> to 29341fbfcc86453da034b2c10e418b3e (127.15.75.130). New cstate: current_term: 1 leader_uuid: "29341fbfcc86453da034b2c10e418b3e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29341fbfcc86453da034b2c10e418b3e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45381 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4ba2d99b0bd74c71b736bed8f281e453" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8503d46e570848f0af6333917d9fee07" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 40471 } health_report { overall_health: UNKNOWN } } }
W20260809 02:19:14.903612 7923 tablet.cc:2369] T 1a3b6ba630cc434d8898d43027e70e07 P 8503d46e570848f0af6333917d9fee07: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:19:14.904161 8356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:14.900652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34390 (local address 127.15.75.130:45381)
0809 02:19:14.900773 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:14.900779 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:14.900825 (+ 46us) server_negotiation.cc:407] Connection header received
0809 02:19:14.901004 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:14.901007 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:14.901056 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:14.901157 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:19:14.901401 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:14.901869 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:14.902724 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:14.902937 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:14.903040 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:14.903327 (+ 287us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:19:14.903802 (+ 475us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:19:14.903919 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:14.903999 (+ 80us) server_negotiation.cc:299] Negotiation successful
0809 02:19:14.904055 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 02:19:14.909312 7853 raft_consensus.cc:1261] T 1a3b6ba630cc434d8898d43027e70e07 P 8503d46e570848f0af6333917d9fee07 [term 1 FOLLOWER]: Refusing update from remote peer 29341fbfcc86453da034b2c10e418b3e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:19:14.909384 8268 raft_consensus.cc:1261] T 1a3b6ba630cc434d8898d43027e70e07 P 4ba2d99b0bd74c71b736bed8f281e453 [term 1 FOLLOWER]: Refusing update from remote peer 29341fbfcc86453da034b2c10e418b3e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:19:14.909945 8367 consensus_queue.cc:1035] T 1a3b6ba630cc434d8898d43027e70e07 P 29341fbfcc86453da034b2c10e418b3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8503d46e570848f0af6333917d9fee07" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 40471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:19:14.910125 8371 consensus_queue.cc:1035] T 1a3b6ba630cc434d8898d43027e70e07 P 29341fbfcc86453da034b2c10e418b3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ba2d99b0bd74c71b736bed8f281e453" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:19:14.917047 8376 mvcc.cc:205] Tried to move back new op lower bound from 7316447047302348800 to 7316447047232471040. Current Snapshot: MvccSnapshot[applied={T|T < 7316447047302348800}]
I20260809 02:19:14.917449 8375 mvcc.cc:205] Tried to move back new op lower bound from 7316447047302348800 to 7316447047232471040. Current Snapshot: MvccSnapshot[applied={T|T < 7316447047302348800}]
I20260809 02:19:14.917966 8374 mvcc.cc:205] Tried to move back new op lower bound from 7316447047302348800 to 7316447047232471040. Current Snapshot: MvccSnapshot[applied={T|T < 7316447047302348800}]
I20260809 02:19:14.925748 7639 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56970:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:19:14.925922 7639 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56970:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:19:14.928071 7639 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a708a9aa21ba4915b9ae3f1ff4de527c: Sending DeleteTablet for 3 replicas of tablet 1a3b6ba630cc434d8898d43027e70e07
I20260809 02:19:14.928808 7830 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a3b6ba630cc434d8898d43027e70e07 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:19:14 UTC) from {username='kudu', principal='kudu/127.15.75.190@KRBTEST.COM'} at 127.0.0.1:51766
I20260809 02:19:14.928897 8037 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a3b6ba630cc434d8898d43027e70e07 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:19:14 UTC) from {username='kudu', principal='kudu/127.15.75.190@KRBTEST.COM'} at 127.0.0.1:34374
I20260809 02:19:14.929013 7937 tablet_replica.cc:330] T 1a3b6ba630cc434d8898d43027e70e07 P 8503d46e570848f0af6333917d9fee07: stopping tablet replica
I20260809 02:19:14.929140 7937 raft_consensus.cc:2229] T 1a3b6ba630cc434d8898d43027e70e07 P 8503d46e570848f0af6333917d9fee07 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:19:14.929276 7937 raft_consensus.cc:2258] T 1a3b6ba630cc434d8898d43027e70e07 P 8503d46e570848f0af6333917d9fee07 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:19:14.929589 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 7702
I20260809 02:19:14.929539 8236 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a3b6ba630cc434d8898d43027e70e07 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:19:14 UTC) from {username='kudu', principal='kudu/127.15.75.190@KRBTEST.COM'} at 127.0.0.1:39560
I20260809 02:19:14.937878 8387 tablet_replica.cc:330] T 1a3b6ba630cc434d8898d43027e70e07 P 4ba2d99b0bd74c71b736bed8f281e453: stopping tablet replica
I20260809 02:19:14.938246 8387 raft_consensus.cc:2229] T 1a3b6ba630cc434d8898d43027e70e07 P 4ba2d99b0bd74c71b736bed8f281e453 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:19:14.938568 8387 raft_consensus.cc:2258] T 1a3b6ba630cc434d8898d43027e70e07 P 4ba2d99b0bd74c71b736bed8f281e453 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:19:14.940843 8387 ts_tablet_manager.cc:1905] T 1a3b6ba630cc434d8898d43027e70e07 P 4ba2d99b0bd74c71b736bed8f281e453: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:19:14.940979 8386 tablet_replica.cc:330] T 1a3b6ba630cc434d8898d43027e70e07 P 29341fbfcc86453da034b2c10e418b3e: stopping tablet replica
I20260809 02:19:14.941253 8386 raft_consensus.cc:2229] T 1a3b6ba630cc434d8898d43027e70e07 P 29341fbfcc86453da034b2c10e418b3e [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:19:14.941675 8386 raft_consensus.cc:2258] T 1a3b6ba630cc434d8898d43027e70e07 P 29341fbfcc86453da034b2c10e418b3e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:19:14.943092 8386 ts_tablet_manager.cc:1905] T 1a3b6ba630cc434d8898d43027e70e07 P 29341fbfcc86453da034b2c10e418b3e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:19:14.943853 8387 ts_tablet_manager.cc:1918] T 1a3b6ba630cc434d8898d43027e70e07 P 4ba2d99b0bd74c71b736bed8f281e453: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:19:14.943948 8387 log.cc:1196] T 1a3b6ba630cc434d8898d43027e70e07 P 4ba2d99b0bd74c71b736bed8f281e453: Deleting WAL directory at /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-2/wal/wals/1a3b6ba630cc434d8898d43027e70e07
I20260809 02:19:14.944299 8387 ts_tablet_manager.cc:1939] T 1a3b6ba630cc434d8898d43027e70e07 P 4ba2d99b0bd74c71b736bed8f281e453: Deleting consensus metadata
I20260809 02:19:14.945730 7619 catalog_manager.cc:4841] TS 4ba2d99b0bd74c71b736bed8f281e453 (127.15.75.131:45603): tablet 1a3b6ba630cc434d8898d43027e70e07 (table test-table [id=1a2080538d2e4967b9b51d849ea21591]) successfully deleted
I20260809 02:19:14.945920 8386 ts_tablet_manager.cc:1918] T 1a3b6ba630cc434d8898d43027e70e07 P 29341fbfcc86453da034b2c10e418b3e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:19:14.946008 8386 log.cc:1196] T 1a3b6ba630cc434d8898d43027e70e07 P 29341fbfcc86453da034b2c10e418b3e: Deleting WAL directory at /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-1/wal/wals/1a3b6ba630cc434d8898d43027e70e07
I20260809 02:19:14.946282 8386 ts_tablet_manager.cc:1939] T 1a3b6ba630cc434d8898d43027e70e07 P 29341fbfcc86453da034b2c10e418b3e: Deleting consensus metadata
W20260809 02:19:14.946758 7619 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.75.129:40471 (error 108)
I20260809 02:19:14.947330 7615 catalog_manager.cc:4841] TS 29341fbfcc86453da034b2c10e418b3e (127.15.75.130:45381): tablet 1a3b6ba630cc434d8898d43027e70e07 (table test-table [id=1a2080538d2e4967b9b51d849ea21591]) successfully deleted
W20260809 02:19:14.948269 8359 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:19:14.947544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.129:40471 (local address 127.0.0.1:51778)
0809 02:19:14.947858 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:14.947874 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 02:19:14.948004 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:19:14.948194 (+ 190us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.129:40471: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":224}
I20260809 02:19:14.948546 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 7931
W20260809 02:19:14.948500 7619 catalog_manager.cc:4584] TS 8503d46e570848f0af6333917d9fee07 (127.15.75.129:40471): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1a3b6ba630cc434d8898d43027e70e07: Network error: Client connection negotiation failed: client connection to 127.15.75.129:40471: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 02:19:14.964764 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 8134
I20260809 02:19:14.976749 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 7588
I20260809 02:19:34.986865 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:36755
--webserver_interface=127.15.75.190
--webserver_port=36907
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab
--principal=kudu/127.15.75.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:45655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.75.190:36755
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:19:35.098114 8391 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:19:35.098385 8391 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:19:35.098452 8391 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:19:35.102435 8391 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 02:19:35.102502 8391 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 02:19:35.102528 8391 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 02:19:35.102558 8391 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:19:35.102602 8391 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:19:35.102639 8391 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:19:35.102720 8391 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:19:35.102756 8391 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:19:35.107769 8391 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:45655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.15.75.190:36755
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab
--principal=kudu/127.15.75.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:36755
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.75.190
--webserver_port=36907
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8391
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:19:35.108784 8391 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:19:35.109747 8391 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:19:35.118554 8400 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:19:35.118605 8391 server_base.cc:1034] running on GCE node
W20260809 02:19:35.118642 8401 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:19:35.118562 8405 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:19:35.119396 8391 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:19:35.120394 8391 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:19:35.121554 8391 hybrid_clock.cc:648] HybridClock initialized: now 1786241975121522 us; error 44 us; skew 500 ppm
Aug 09 02:19:35 dist-test-slave-s11t krb5kdc[6753](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241975, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:19:35.124228 8391 init.cc:377] Logged in from keytab as kudu/127.15.75.190@KRBTEST.COM (short username kudu)
I20260809 02:19:35.125166 8391 webserver.cc:460] Webserver started at http://127.15.75.190:36907/ using document root <none> and password file <none>
I20260809 02:19:35.125707 8391 fs_manager.cc:359] Metadata directory not provided
I20260809 02:19:35.125799 8391 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:19:35.129407 8391 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260809 02:19:35.131386 8418 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:19:35.132570 8391 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 02:19:35.132721 8391 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/master-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/master-0/wal
uuid: "a708a9aa21ba4915b9ae3f1ff4de527c"
format_stamp: "Formatted at 2026-08-09 02:18:52 on dist-test-slave-s11t"
server_key: "5d58d775c4e31ddf3fd36fd7b0e4d8ad"
server_key_iv: "193ea38e7521fe51bb2e9302c4109c7f"
server_key_version: "encryptionkey@0"
I20260809 02:19:35.133252 8391 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:19:35.145223 8391 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:19:35.147755 8391 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:19:35.147953 8391 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:19:35.155161 8391 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.190:36755
I20260809 02:19:35.155449 8496 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.190:36755 every 8 connection(s)
I20260809 02:19:35.156194 8391 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
I20260809 02:19:35.160250 8497 sys_catalog.cc:263] Verifying existing consensus state
I20260809 02:19:35.161913 8497 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a708a9aa21ba4915b9ae3f1ff4de527c: Bootstrap starting.
I20260809 02:19:35.163875 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 8391
I20260809 02:19:35.164315 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.129:40471
--local_ip_for_outbound_sockets=127.15.75.129
--tserver_master_addrs=127.15.75.190:36755
--webserver_port=38441
--webserver_interface=127.15.75.129
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab
--principal=kudu/127.15.75.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:45655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 02:19:35.179380 8497 log.cc:824] T 00000000000000000000000000000000 P a708a9aa21ba4915b9ae3f1ff4de527c: Log is configured to *not* fsync() on all Append() calls
I20260809 02:19:35.190519 8497 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a708a9aa21ba4915b9ae3f1ff4de527c: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 02:19:35.190886 8497 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a708a9aa21ba4915b9ae3f1ff4de527c: Bootstrap complete.
I20260809 02:19:35.195181 8497 raft_consensus.cc:348] T 00000000000000000000000000000000 P a708a9aa21ba4915b9ae3f1ff4de527c [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a708a9aa21ba4915b9ae3f1ff4de527c" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 36755 } }
I20260809 02:19:35.195716 8497 raft_consensus.cc:729] T 00000000000000000000000000000000 P a708a9aa21ba4915b9ae3f1ff4de527c [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: a708a9aa21ba4915b9ae3f1ff4de527c, State: Initialized, Role: FOLLOWER
I20260809 02:19:35.196247 8497 consensus_queue.cc:260] T 00000000000000000000000000000000 P a708a9aa21ba4915b9ae3f1ff4de527c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a708a9aa21ba4915b9ae3f1ff4de527c" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 36755 } }
I20260809 02:19:35.196403 8497 raft_consensus.cc:388] T 00000000000000000000000000000000 P a708a9aa21ba4915b9ae3f1ff4de527c [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:19:35.196496 8497 raft_consensus.cc:482] T 00000000000000000000000000000000 P a708a9aa21ba4915b9ae3f1ff4de527c [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:19:35.196592 8497 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a708a9aa21ba4915b9ae3f1ff4de527c [term 2 FOLLOWER]: Advancing to term 3
I20260809 02:19:35.198410 8497 raft_consensus.cc:504] T 00000000000000000000000000000000 P a708a9aa21ba4915b9ae3f1ff4de527c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a708a9aa21ba4915b9ae3f1ff4de527c" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 36755 } }
I20260809 02:19:35.198733 8497 leader_election.cc:302] T 00000000000000000000000000000000 P a708a9aa21ba4915b9ae3f1ff4de527c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a708a9aa21ba4915b9ae3f1ff4de527c; no voters:
I20260809 02:19:35.199064 8497 leader_election.cc:288] T 00000000000000000000000000000000 P a708a9aa21ba4915b9ae3f1ff4de527c [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 02:19:35.199142 8502 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a708a9aa21ba4915b9ae3f1ff4de527c [term 3 FOLLOWER]: Leader election won for term 3
I20260809 02:19:35.199348 8502 raft_consensus.cc:686] T 00000000000000000000000000000000 P a708a9aa21ba4915b9ae3f1ff4de527c [term 3 LEADER]: Becoming Leader. State: Replica: a708a9aa21ba4915b9ae3f1ff4de527c, State: Running, Role: LEADER
I20260809 02:19:35.199680 8502 consensus_queue.cc:237] T 00000000000000000000000000000000 P a708a9aa21ba4915b9ae3f1ff4de527c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a708a9aa21ba4915b9ae3f1ff4de527c" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 36755 } }
I20260809 02:19:35.200277 8497 sys_catalog.cc:564] T 00000000000000000000000000000000 P a708a9aa21ba4915b9ae3f1ff4de527c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:19:35.200592 8502 sys_catalog.cc:455] T 00000000000000000000000000000000 P a708a9aa21ba4915b9ae3f1ff4de527c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "a708a9aa21ba4915b9ae3f1ff4de527c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a708a9aa21ba4915b9ae3f1ff4de527c" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 36755 } } }
I20260809 02:19:35.200748 8502 sys_catalog.cc:458] T 00000000000000000000000000000000 P a708a9aa21ba4915b9ae3f1ff4de527c [sys.catalog]: This master's current role is: LEADER
I20260809 02:19:35.201172 8503 sys_catalog.cc:455] T 00000000000000000000000000000000 P a708a9aa21ba4915b9ae3f1ff4de527c [sys.catalog]: SysCatalogTable state changed. Reason: New leader a708a9aa21ba4915b9ae3f1ff4de527c. Latest consensus state: current_term: 3 leader_uuid: "a708a9aa21ba4915b9ae3f1ff4de527c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a708a9aa21ba4915b9ae3f1ff4de527c" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 36755 } } }
I20260809 02:19:35.201337 8503 sys_catalog.cc:458] T 00000000000000000000000000000000 P a708a9aa21ba4915b9ae3f1ff4de527c [sys.catalog]: This master's current role is: LEADER
I20260809 02:19:35.201651 8509 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:19:35.206617 8509 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:19:35.207327 8509 catalog_manager.cc:1259] Loaded cluster ID: 60c1e4383e294c28931f3fe91c6df02b
I20260809 02:19:35.207387 8509 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:19:35.209854 8509 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:19:35.210731 8509 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a708a9aa21ba4915b9ae3f1ff4de527c: Loaded TSK: 0
I20260809 02:19:35.210830 8509 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a708a9aa21ba4915b9ae3f1ff4de527c: Loaded TSK: 1
F20260809 02:19:35.211700 8509 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f60a20b09ad google::LogMessage::Fail() at ??:0
@ 0x7f60a20b388a google::LogMessage::SendToLog() at ??:0
@ 0x7f60a20b04de google::LogMessage::Flush() at ??:0
@ 0x7f60a20b1a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f60a4d7418d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f60a4d6cf0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f60aca157bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f60ac9ef99c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f60ac9f0178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7f60aca31df8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f60aca5082b std::function<>::operator()() at ??:0
@ 0x7f60ac9efb4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f60ac9f0f3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f60ac9ed579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f60aca31640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f60a43023d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f60a4302c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f60a43043f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f60a42f3512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f60a59376db start_thread at ??:0
@ 0x7f60a0bae61f clone at ??:0
W20260809 02:19:35.314826 8499 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:19:35.315066 8499 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:19:35.315109 8499 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:19:35.319114 8499 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:19:35.319167 8499 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:19:35.319254 8499 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.129
I20260809 02:19:35.324432 8499 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:45655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab
--principal=kudu/127.15.75.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.129:40471
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.75.129
--webserver_port=38441
--tserver_master_addrs=127.15.75.190:36755
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8499
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.129
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:19:35.325477 8499 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:19:35.326479 8499 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:19:35.335913 8587 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:19:35.335929 8592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:19:35.335947 8586 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:19:35.336993 8499 server_base.cc:1034] running on GCE node
I20260809 02:19:35.337390 8499 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:19:35.338001 8499 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:19:35.339190 8499 hybrid_clock.cc:648] HybridClock initialized: now 1786241975339165 us; error 45 us; skew 500 ppm
Aug 09 02:19:35 dist-test-slave-s11t krb5kdc[6753](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241975, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:19:35.342103 8499 init.cc:377] Logged in from keytab as kudu/127.15.75.129@KRBTEST.COM (short username kudu)
I20260809 02:19:35.342988 8499 webserver.cc:460] Webserver started at http://127.15.75.129:38441/ using document root <none> and password file <none>
I20260809 02:19:35.343545 8499 fs_manager.cc:359] Metadata directory not provided
I20260809 02:19:35.343639 8499 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:19:35.347592 8499 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260809 02:19:35.349921 8601 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:19:35.351238 8499 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 02:19:35.351377 8499 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-0/wal
uuid: "8503d46e570848f0af6333917d9fee07"
format_stamp: "Formatted at 2026-08-09 02:18:53 on dist-test-slave-s11t"
server_key: "08b453b8585072939f4b0089a3953574"
server_key_iv: "2f4271788fd852c24df45b4202c41058"
server_key_version: "encryptionkey@0"
I20260809 02:19:35.351922 8499 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:19:35.387893 8499 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:19:35.391170 8499 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:19:35.391382 8499 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:19:35.392017 8499 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:19:35.393263 8615 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 02:19:35.398732 8499 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 02:19:35.398828 8499 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260809 02:19:35.398926 8499 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 02:19:35.400544 8499 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 02:19:35.400615 8499 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260809 02:19:35.400959 8615 tablet_bootstrap.cc:492] T 1a3b6ba630cc434d8898d43027e70e07 P 8503d46e570848f0af6333917d9fee07: Bootstrap starting.
I20260809 02:19:35.411576 8499 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.129:40471
I20260809 02:19:35.411613 8787 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.129:40471 every 8 connection(s)
I20260809 02:19:35.412607 8499 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
I20260809 02:19:35.422516 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 8499
I20260809 02:19:35.422968 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.130:45381
--local_ip_for_outbound_sockets=127.15.75.130
--tserver_master_addrs=127.15.75.190:36755
--webserver_port=43373
--webserver_interface=127.15.75.130
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab
--principal=kudu/127.15.75.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:45655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 02:19:35.423033 8615 log.cc:824] T 1a3b6ba630cc434d8898d43027e70e07 P 8503d46e570848f0af6333917d9fee07: Log is configured to *not* fsync() on all Append() calls
I20260809 02:19:35.432204 8615 tablet_bootstrap.cc:492] T 1a3b6ba630cc434d8898d43027e70e07 P 8503d46e570848f0af6333917d9fee07: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 02:19:35.432726 8615 tablet_bootstrap.cc:492] T 1a3b6ba630cc434d8898d43027e70e07 P 8503d46e570848f0af6333917d9fee07: Bootstrap complete.
I20260809 02:19:35.434218 8615 ts_tablet_manager.cc:1397] T 1a3b6ba630cc434d8898d43027e70e07 P 8503d46e570848f0af6333917d9fee07: Time spent bootstrapping tablet: real 0.034s user 0.020s sys 0.011s
I20260809 02:19:35.437704 8615 raft_consensus.cc:348] T 1a3b6ba630cc434d8898d43027e70e07 P 8503d46e570848f0af6333917d9fee07 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29341fbfcc86453da034b2c10e418b3e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45381 } } peers { permanent_uuid: "4ba2d99b0bd74c71b736bed8f281e453" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45603 } } peers { permanent_uuid: "8503d46e570848f0af6333917d9fee07" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 40471 } }
I20260809 02:19:35.438383 8615 raft_consensus.cc:729] T 1a3b6ba630cc434d8898d43027e70e07 P 8503d46e570848f0af6333917d9fee07 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8503d46e570848f0af6333917d9fee07, State: Initialized, Role: FOLLOWER
I20260809 02:19:35.445747 8615 consensus_queue.cc:260] T 1a3b6ba630cc434d8898d43027e70e07 P 8503d46e570848f0af6333917d9fee07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29341fbfcc86453da034b2c10e418b3e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45381 } } peers { permanent_uuid: "4ba2d99b0bd74c71b736bed8f281e453" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45603 } } peers { permanent_uuid: "8503d46e570848f0af6333917d9fee07" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 40471 } }
I20260809 02:19:35.446869 8615 ts_tablet_manager.cc:1428] T 1a3b6ba630cc434d8898d43027e70e07 P 8503d46e570848f0af6333917d9fee07: Time spent starting tablet: real 0.013s user 0.006s sys 0.001s
W20260809 02:19:35.452466 8791 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:19:35.414566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36755 (local address 127.15.75.129:45059)
0809 02:19:35.415048 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:35.415101 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 02:19:35.415910 (+ 809us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:19:35.451593 (+ 35683us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:36755: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":136,"threads_started":1}
W20260809 02:19:35.453056 8789 heartbeater.cc:644] Failed to heartbeat to 127.15.75.190:36755 (0 consecutive failures): Network error: Failed to ping master at 127.15.75.190:36755: Client connection negotiation failed: client connection to 127.15.75.190:36755: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 02:19:35.453768 8791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:35.453491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36755 (local address 127.15.75.129:48209)
0809 02:19:35.453630 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:35.453703 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:36755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260809 02:19:35.454447 8791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:35.454227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36755 (local address 127.15.75.129:55263)
0809 02:19:35.454337 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:35.454388 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:36755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260809 02:19:35.454660 8789 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 02:19:35.538468 8793 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:19:35.538723 8793 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:19:35.538791 8793 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:19:35.542753 8793 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:19:35.542831 8793 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:19:35.542915 8793 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.130
I20260809 02:19:35.547943 8793 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:45655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab
--principal=kudu/127.15.75.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.130:45381
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.75.130
--webserver_port=43373
--tserver_master_addrs=127.15.75.190:36755
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8793
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.130
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:19:35.548916 8793 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:19:35.549894 8793 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:19:35.559175 8805 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:19:35.559190 8811 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:19:35.559185 8808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:19:35.559517 8793 server_base.cc:1034] running on GCE node
I20260809 02:19:35.559945 8793 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:19:35.560549 8793 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:19:35.561691 8793 hybrid_clock.cc:648] HybridClock initialized: now 1786241975561664 us; error 51 us; skew 500 ppm
Aug 09 02:19:35 dist-test-slave-s11t krb5kdc[6753](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241975, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:19:35.564415 8793 init.cc:377] Logged in from keytab as kudu/127.15.75.130@KRBTEST.COM (short username kudu)
I20260809 02:19:35.565246 8793 webserver.cc:460] Webserver started at http://127.15.75.130:43373/ using document root <none> and password file <none>
I20260809 02:19:35.565752 8793 fs_manager.cc:359] Metadata directory not provided
I20260809 02:19:35.565810 8793 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:19:35.569335 8793 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260809 02:19:35.571254 8819 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:19:35.572423 8793 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 02:19:35.572552 8793 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-1/wal
uuid: "29341fbfcc86453da034b2c10e418b3e"
format_stamp: "Formatted at 2026-08-09 02:18:53 on dist-test-slave-s11t"
server_key: "2f512ec72d10f1d90b5bfcd88748de8d"
server_key_iv: "b6d6f1df53f3364551c64d3ec5f66f4e"
server_key_version: "encryptionkey@0"
I20260809 02:19:35.573120 8793 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:19:35.585760 8793 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:19:35.588891 8793 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:19:35.589077 8793 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:19:35.589634 8793 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:19:35.590436 8793 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:19:35.590484 8793 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:19:35.590521 8793 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:19:35.590538 8793 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:19:35.599366 8793 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.130:45381
I20260809 02:19:35.599442 8979 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.130:45381 every 8 connection(s)
I20260809 02:19:35.600436 8793 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
I20260809 02:19:35.604218 8982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:35.602516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36755 (local address 127.15.75.130:33515)
0809 02:19:35.602995 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:35.603097 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:36755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":136,"threads_started":1}
W20260809 02:19:35.604796 8980 heartbeater.cc:644] Failed to heartbeat to 127.15.75.190:36755 (0 consecutive failures): Network error: Failed to ping master at 127.15.75.190:36755: Client connection negotiation failed: client connection to 127.15.75.190:36755: connect: Connection refused (error 111)
I20260809 02:19:35.605353 8982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:35.605154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36755 (local address 127.15.75.130:33025)
0809 02:19:35.605257 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:35.605302 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:36755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260809 02:19:35.605916 8982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:35.605726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36755 (local address 127.15.75.130:50481)
0809 02:19:35.605820 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:35.605860 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:36755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260809 02:19:35.606076 8980 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 02:19:35.610208 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 8793
I20260809 02:19:35.610594 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.131:45603
--local_ip_for_outbound_sockets=127.15.75.131
--tserver_master_addrs=127.15.75.190:36755
--webserver_port=36691
--webserver_interface=127.15.75.131
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab
--principal=kudu/127.15.75.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:45655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:19:35.721297 8983 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:19:35.721575 8983 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:19:35.721652 8983 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:19:35.725557 8983 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:19:35.725641 8983 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:19:35.725731 8983 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.131
I20260809 02:19:35.730670 8983 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:45655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab
--principal=kudu/127.15.75.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.131:45603
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.75.131
--webserver_port=36691
--tserver_master_addrs=127.15.75.190:36755
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8983
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.131
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:19:35.731603 8983 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:19:35.732671 8983 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:19:35.742318 8994 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:19:35.742364 8993 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:19:35.742528 8996 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:19:35.742568 8983 server_base.cc:1034] running on GCE node
I20260809 02:19:35.742990 8983 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:19:35.743574 8983 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:19:35.744768 8983 hybrid_clock.cc:648] HybridClock initialized: now 1786241975744742 us; error 48 us; skew 500 ppm
Aug 09 02:19:35 dist-test-slave-s11t krb5kdc[6753](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241975, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:19:35.747398 8983 init.cc:377] Logged in from keytab as kudu/127.15.75.131@KRBTEST.COM (short username kudu)
I20260809 02:19:35.748274 8983 webserver.cc:460] Webserver started at http://127.15.75.131:36691/ using document root <none> and password file <none>
I20260809 02:19:35.748905 8983 fs_manager.cc:359] Metadata directory not provided
I20260809 02:19:35.748999 8983 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:19:35.752576 8983 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260809 02:19:35.754882 9008 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:19:35.756448 8983 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 02:19:35.756556 8983 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-2/wal
uuid: "4ba2d99b0bd74c71b736bed8f281e453"
format_stamp: "Formatted at 2026-08-09 02:18:53 on dist-test-slave-s11t"
server_key: "2d5bef98351dd6812a39e5b7901a1654"
server_key_iv: "9baa0bb5e138d4a88823d87de26ce886"
server_key_version: "encryptionkey@0"
I20260809 02:19:35.757125 8983 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:19:35.768651 8983 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:19:35.771837 8983 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:19:35.771986 8983 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:19:35.772490 8983 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:19:35.773262 8983 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:19:35.773309 8983 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:19:35.773348 8983 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:19:35.773378 8983 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:19:35.782341 8983 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.131:45603
I20260809 02:19:35.782414 9180 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.131:45603 every 8 connection(s)
I20260809 02:19:35.783352 8983 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
I20260809 02:19:35.787009 9184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:35.785332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36755 (local address 127.15.75.131:54145)
0809 02:19:35.785783 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:35.785899 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:36755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":119,"threads_started":1}
I20260809 02:19:35.787324 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 8983
W20260809 02:19:35.787536 9182 heartbeater.cc:644] Failed to heartbeat to 127.15.75.190:36755 (0 consecutive failures): Network error: Failed to ping master at 127.15.75.190:36755: Client connection negotiation failed: client connection to 127.15.75.190:36755: connect: Connection refused (error 111)
I20260809 02:19:35.788098 9184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:35.787881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36755 (local address 127.15.75.131:51031)
0809 02:19:35.787999 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:35.788046 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:36755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260809 02:19:35.788743 9184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:35.788510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36755 (local address 127.15.75.131:42541)
0809 02:19:35.788608 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:35.788652 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:36755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260809 02:19:35.788928 9182 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 02:19:36.455633 9188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:36.455229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36755 (local address 127.15.75.129:47485)
0809 02:19:36.455470 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:36.455522 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:36755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":74,"threads_started":1}
I20260809 02:19:36.606938 9189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:36.606553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36755 (local address 127.15.75.130:50423)
0809 02:19:36.606789 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:36.606842 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:36755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":68,"threads_started":1}
I20260809 02:19:36.643520 9190 raft_consensus.cc:482] T 1a3b6ba630cc434d8898d43027e70e07 P 8503d46e570848f0af6333917d9fee07 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:19:36.643749 9190 raft_consensus.cc:504] T 1a3b6ba630cc434d8898d43027e70e07 P 8503d46e570848f0af6333917d9fee07 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29341fbfcc86453da034b2c10e418b3e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45381 } } peers { permanent_uuid: "4ba2d99b0bd74c71b736bed8f281e453" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45603 } } peers { permanent_uuid: "8503d46e570848f0af6333917d9fee07" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 40471 } }
I20260809 02:19:36.644932 9190 leader_election.cc:288] T 1a3b6ba630cc434d8898d43027e70e07 P 8503d46e570848f0af6333917d9fee07 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 29341fbfcc86453da034b2c10e418b3e (127.15.75.130:45381), 4ba2d99b0bd74c71b736bed8f281e453 (127.15.75.131:45603)
Aug 09 02:19:36 dist-test-slave-s11t krb5kdc[6753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241975, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.129@KRBTEST.COM for kudu/127.15.75.131@KRBTEST.COM
Aug 09 02:19:36 dist-test-slave-s11t krb5kdc[6753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241975, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.129@KRBTEST.COM for kudu/127.15.75.130@KRBTEST.COM
I20260809 02:19:36.662774 9188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:36.645071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.130:45381 (local address 127.15.75.129:48725)
0809 02:19:36.645217 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:36.645234 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 02:19:36.645344 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:19:36.647433 (+ 2089us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:19:36.647438 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:19:36.647788 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:19:36.648224 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:36.648237 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:36.649612 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:36.649618 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:36.650534 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:36.650544 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:36.651248 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:36.651254 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:36.651438 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:36.656304 (+ 4866us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:19:36.656316 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:19:36.658363 (+ 2047us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:19:36.660629 (+ 2266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:19:36.660633 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:19:36.660635 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:19:36.660907 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:19:36.661246 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:19:36.661250 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:19:36.661252 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:19:36.661341 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:19:36.661985 (+ 644us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:19:36.661989 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:19:36.662084 (+ 95us) client_negotiation.cc:769] Sending connection context
0809 02:19:36.662229 (+ 145us) client_negotiation.cc:240] Negotiation successful
0809 02:19:36.662575 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60,"mutex_wait_us":38}
I20260809 02:19:36.662787 9193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:36.645239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.129:48725 (local address 127.15.75.130:45381)
0809 02:19:36.645577 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:36.645623 (+ 46us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:36.645672 (+ 49us) server_negotiation.cc:407] Connection header received
0809 02:19:36.646597 (+ 925us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:36.646605 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:36.647046 (+ 441us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:36.647376 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:36.648420 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:36.649462 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:36.650664 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:36.651128 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:36.651255 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:36.658585 (+ 7330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:19:36.658592 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:19:36.658608 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:19:36.658643 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:19:36.660481 (+ 1838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:36.661014 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:36.661020 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:36.661031 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:36.661141 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:36.661445 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:36.661449 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:36.661451 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:36.661873 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:19:36.662013 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:36.662345 (+ 332us) server_negotiation.cc:299] Negotiation successful
0809 02:19:36.662596 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":224,"thread_start_us":90,"threads_started":1}
I20260809 02:19:36.662827 9191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:36.645206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.131:45603 (local address 127.15.75.129:49925)
0809 02:19:36.645601 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:36.645615 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:19:36.645710 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:19:36.647857 (+ 2147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:19:36.647859 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:19:36.648021 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:19:36.648223 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:36.648238 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:36.649645 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:36.649647 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:36.650537 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:36.650548 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:36.651456 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:36.651459 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:36.651521 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:36.656278 (+ 4757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:19:36.656307 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:19:36.658004 (+ 1697us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:19:36.660259 (+ 2255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:19:36.660265 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:19:36.660279 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:19:36.660534 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:19:36.661022 (+ 488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:19:36.661024 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:19:36.661025 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:19:36.661152 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:19:36.661825 (+ 673us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:19:36.661831 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:19:36.662047 (+ 216us) client_negotiation.cc:769] Sending connection context
0809 02:19:36.662230 (+ 183us) client_negotiation.cc:240] Negotiation successful
0809 02:19:36.662657 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"mutex_wait_us":240,"spinlock_wait_cycles":13824,"thread_start_us":86,"threads_started":1}
I20260809 02:19:36.663410 9194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:36.645430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.129:49925 (local address 127.15.75.131:45603)
0809 02:19:36.645810 (+ 380us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:36.645822 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:36.645856 (+ 34us) server_negotiation.cc:407] Connection header received
0809 02:19:36.646596 (+ 740us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:36.646605 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:36.647046 (+ 441us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:36.647376 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:36.648426 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:36.649478 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:36.650691 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:36.651309 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:36.651457 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:36.658245 (+ 6788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:19:36.658250 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:19:36.658262 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:19:36.658288 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:19:36.660137 (+ 1849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:36.660675 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:36.660718 (+ 43us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:36.660724 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:36.660845 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:36.661255 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:36.661261 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:36.661265 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:36.661701 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:19:36.661851 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:36.662344 (+ 493us) server_negotiation.cc:299] Negotiation successful
0809 02:19:36.662631 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":128,"threads_started":1}
W20260809 02:19:36.665545 8603 leader_election.cc:341] T 1a3b6ba630cc434d8898d43027e70e07 P 8503d46e570848f0af6333917d9fee07 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 29341fbfcc86453da034b2c10e418b3e (127.15.75.130:45381): Not found: Tablet not found: 1a3b6ba630cc434d8898d43027e70e07
W20260809 02:19:36.665849 8605 leader_election.cc:341] T 1a3b6ba630cc434d8898d43027e70e07 P 8503d46e570848f0af6333917d9fee07 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4ba2d99b0bd74c71b736bed8f281e453 (127.15.75.131:45603): Not found: Tablet not found: 1a3b6ba630cc434d8898d43027e70e07
I20260809 02:19:36.665938 8605 leader_election.cc:302] T 1a3b6ba630cc434d8898d43027e70e07 P 8503d46e570848f0af6333917d9fee07 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8503d46e570848f0af6333917d9fee07; no voters: 29341fbfcc86453da034b2c10e418b3e, 4ba2d99b0bd74c71b736bed8f281e453
I20260809 02:19:36.666203 9190 raft_consensus.cc:2726] T 1a3b6ba630cc434d8898d43027e70e07 P 8503d46e570848f0af6333917d9fee07 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 02:19:36.789759 9196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:36.789400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36755 (local address 127.15.75.131:36687)
0809 02:19:36.789621 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:36.789671 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:36755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":70,"threads_started":1}
I20260809 02:19:37.456969 9197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:37.456468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36755 (local address 127.15.75.129:33031)
0809 02:19:37.456822 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:37.456875 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:36755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":96,"threads_started":1}
I20260809 02:19:37.608039 9198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:37.607659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36755 (local address 127.15.75.130:41629)
0809 02:19:37.607895 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:37.607950 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:36755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":69,"threads_started":1}
I20260809 02:19:37.790870 9201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:37.790468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36755 (local address 127.15.75.131:53429)
0809 02:19:37.790721 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:37.790772 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:36755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":77,"threads_started":1}
I20260809 02:19:38.458068 9202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:38.457652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36755 (local address 127.15.75.129:52389)
0809 02:19:38.457920 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:38.457973 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:36755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":92,"threads_started":1}
I20260809 02:19:38.547577 9203 raft_consensus.cc:482] T 1a3b6ba630cc434d8898d43027e70e07 P 8503d46e570848f0af6333917d9fee07 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:19:38.547703 9203 raft_consensus.cc:504] T 1a3b6ba630cc434d8898d43027e70e07 P 8503d46e570848f0af6333917d9fee07 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29341fbfcc86453da034b2c10e418b3e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45381 } } peers { permanent_uuid: "4ba2d99b0bd74c71b736bed8f281e453" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45603 } } peers { permanent_uuid: "8503d46e570848f0af6333917d9fee07" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 40471 } }
I20260809 02:19:38.548117 9203 leader_election.cc:288] T 1a3b6ba630cc434d8898d43027e70e07 P 8503d46e570848f0af6333917d9fee07 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 29341fbfcc86453da034b2c10e418b3e (127.15.75.130:45381), 4ba2d99b0bd74c71b736bed8f281e453 (127.15.75.131:45603)
W20260809 02:19:38.548977 8603 leader_election.cc:341] T 1a3b6ba630cc434d8898d43027e70e07 P 8503d46e570848f0af6333917d9fee07 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 29341fbfcc86453da034b2c10e418b3e (127.15.75.130:45381): Not found: Tablet not found: 1a3b6ba630cc434d8898d43027e70e07
W20260809 02:19:38.549136 8605 leader_election.cc:341] T 1a3b6ba630cc434d8898d43027e70e07 P 8503d46e570848f0af6333917d9fee07 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4ba2d99b0bd74c71b736bed8f281e453 (127.15.75.131:45603): Not found: Tablet not found: 1a3b6ba630cc434d8898d43027e70e07
I20260809 02:19:38.549226 8605 leader_election.cc:302] T 1a3b6ba630cc434d8898d43027e70e07 P 8503d46e570848f0af6333917d9fee07 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8503d46e570848f0af6333917d9fee07; no voters: 29341fbfcc86453da034b2c10e418b3e, 4ba2d99b0bd74c71b736bed8f281e453
I20260809 02:19:38.549405 9203 raft_consensus.cc:2726] T 1a3b6ba630cc434d8898d43027e70e07 P 8503d46e570848f0af6333917d9fee07 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 02:19:38.609153 9204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:38.608780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36755 (local address 127.15.75.130:49609)
0809 02:19:38.609016 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:38.609069 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:36755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":72,"threads_started":1}
I20260809 02:19:38.791972 9205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:38.791570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36755 (local address 127.15.75.131:52775)
0809 02:19:38.791836 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:38.791888 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:36755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":74,"threads_started":1}
I20260809 02:19:39.459194 9209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:39.458792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36755 (local address 127.15.75.129:53279)
0809 02:19:39.459045 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:39.459098 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:36755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":80,"threads_started":1}
I20260809 02:19:39.610272 9210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:39.609874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36755 (local address 127.15.75.130:51961)
0809 02:19:39.610129 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:39.610182 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:36755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":92,"threads_started":1}
I20260809 02:19:39.793124 9211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:39.792665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36755 (local address 127.15.75.131:36257)
0809 02:19:39.792984 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:39.793037 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:36755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":74,"threads_started":1}
I20260809 02:19:40.460549 9212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:40.460068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36755 (local address 127.15.75.129:60619)
0809 02:19:40.460399 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:40.460454 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:36755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":74,"threads_started":1}
I20260809 02:19:40.611443 9213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:40.611059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36755 (local address 127.15.75.130:55685)
0809 02:19:40.611308 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:40.611360 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:36755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":94,"threads_started":1}
I20260809 02:19:40.794258 9214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:40.793859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36755 (local address 127.15.75.131:39791)
0809 02:19:40.794121 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:40.794173 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:36755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":102,"threads_started":1}
I20260809 02:19:40.863468 9216 raft_consensus.cc:482] T 1a3b6ba630cc434d8898d43027e70e07 P 8503d46e570848f0af6333917d9fee07 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:19:40.863615 9216 raft_consensus.cc:504] T 1a3b6ba630cc434d8898d43027e70e07 P 8503d46e570848f0af6333917d9fee07 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29341fbfcc86453da034b2c10e418b3e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45381 } } peers { permanent_uuid: "4ba2d99b0bd74c71b736bed8f281e453" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45603 } } peers { permanent_uuid: "8503d46e570848f0af6333917d9fee07" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 40471 } }
I20260809 02:19:40.864032 9216 leader_election.cc:288] T 1a3b6ba630cc434d8898d43027e70e07 P 8503d46e570848f0af6333917d9fee07 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 29341fbfcc86453da034b2c10e418b3e (127.15.75.130:45381), 4ba2d99b0bd74c71b736bed8f281e453 (127.15.75.131:45603)
W20260809 02:19:40.864871 8605 leader_election.cc:341] T 1a3b6ba630cc434d8898d43027e70e07 P 8503d46e570848f0af6333917d9fee07 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4ba2d99b0bd74c71b736bed8f281e453 (127.15.75.131:45603): Not found: Tablet not found: 1a3b6ba630cc434d8898d43027e70e07
W20260809 02:19:40.864979 8603 leader_election.cc:341] T 1a3b6ba630cc434d8898d43027e70e07 P 8503d46e570848f0af6333917d9fee07 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 29341fbfcc86453da034b2c10e418b3e (127.15.75.130:45381): Not found: Tablet not found: 1a3b6ba630cc434d8898d43027e70e07
I20260809 02:19:40.865051 8603 leader_election.cc:302] T 1a3b6ba630cc434d8898d43027e70e07 P 8503d46e570848f0af6333917d9fee07 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8503d46e570848f0af6333917d9fee07; no voters: 29341fbfcc86453da034b2c10e418b3e, 4ba2d99b0bd74c71b736bed8f281e453
I20260809 02:19:40.865188 9216 raft_consensus.cc:2726] T 1a3b6ba630cc434d8898d43027e70e07 P 8503d46e570848f0af6333917d9fee07 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 02:19:41.461782 9217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:41.461380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36755 (local address 127.15.75.129:57603)
0809 02:19:41.461633 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:41.461685 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:36755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":93,"threads_started":1}
I20260809 02:19:41.612524 9218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:41.612144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36755 (local address 127.15.75.130:56491)
0809 02:19:41.612386 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:41.612441 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:36755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":71,"threads_started":1}
I20260809 02:19:41.795353 9219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:41.794929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36755 (local address 127.15.75.131:41757)
0809 02:19:41.795205 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:41.795269 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:36755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":90,"threads_started":1}
I20260809 02:19:42.462909 9220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:42.462446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36755 (local address 127.15.75.129:34153)
0809 02:19:42.462754 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:42.462810 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:36755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":74,"threads_started":1}
I20260809 02:19:42.613708 9221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:42.613268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36755 (local address 127.15.75.130:35801)
0809 02:19:42.613562 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:42.613617 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:36755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":82,"threads_started":1}
I20260809 02:19:42.796475 9222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:42.796016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36755 (local address 127.15.75.131:57319)
0809 02:19:42.796330 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:42.796386 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:36755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":69,"threads_started":1}
I20260809 02:19:43.464075 9223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:43.463668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36755 (local address 127.15.75.129:57921)
0809 02:19:43.463915 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:43.463979 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:36755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":76,"threads_started":1}
I20260809 02:19:43.614861 9225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:43.614414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36755 (local address 127.15.75.130:56739)
0809 02:19:43.614691 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:43.614763 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:36755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":77,"threads_started":1}
I20260809 02:19:43.797557 9226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:43.797191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36755 (local address 127.15.75.131:44239)
0809 02:19:43.797418 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:43.797470 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:36755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":71,"threads_started":1}
I20260809 02:19:44.465193 9227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:44.464805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36755 (local address 127.15.75.129:43865)
0809 02:19:44.465044 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:44.465097 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:36755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":70,"threads_started":1}
I20260809 02:19:44.616195 9228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:44.615768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36755 (local address 127.15.75.130:51849)
0809 02:19:44.616045 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:44.616103 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:36755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":79,"threads_started":1}
I20260809 02:19:44.798947 9229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:44.798383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36755 (local address 127.15.75.131:41601)
0809 02:19:44.798706 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:44.798786 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:36755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":111,"threads_started":1}
I20260809 02:19:45.466601 9230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:45.466150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36755 (local address 127.15.75.129:34901)
0809 02:19:45.466401 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:45.466456 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:36755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":81,"threads_started":1}
I20260809 02:19:45.617456 9233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:45.617047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36755 (local address 127.15.75.130:46937)
0809 02:19:45.617300 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:45.617352 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:36755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":89,"threads_started":1}
I20260809 02:19:45.667299 9236 raft_consensus.cc:482] T 1a3b6ba630cc434d8898d43027e70e07 P 8503d46e570848f0af6333917d9fee07 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:19:45.667460 9236 raft_consensus.cc:504] T 1a3b6ba630cc434d8898d43027e70e07 P 8503d46e570848f0af6333917d9fee07 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29341fbfcc86453da034b2c10e418b3e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45381 } } peers { permanent_uuid: "4ba2d99b0bd74c71b736bed8f281e453" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45603 } } peers { permanent_uuid: "8503d46e570848f0af6333917d9fee07" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 40471 } }
I20260809 02:19:45.667894 9236 leader_election.cc:288] T 1a3b6ba630cc434d8898d43027e70e07 P 8503d46e570848f0af6333917d9fee07 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 29341fbfcc86453da034b2c10e418b3e (127.15.75.130:45381), 4ba2d99b0bd74c71b736bed8f281e453 (127.15.75.131:45603)
W20260809 02:19:45.668766 8603 leader_election.cc:341] T 1a3b6ba630cc434d8898d43027e70e07 P 8503d46e570848f0af6333917d9fee07 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 29341fbfcc86453da034b2c10e418b3e (127.15.75.130:45381): Not found: Tablet not found: 1a3b6ba630cc434d8898d43027e70e07
W20260809 02:19:45.668877 8605 leader_election.cc:341] T 1a3b6ba630cc434d8898d43027e70e07 P 8503d46e570848f0af6333917d9fee07 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4ba2d99b0bd74c71b736bed8f281e453 (127.15.75.131:45603): Not found: Tablet not found: 1a3b6ba630cc434d8898d43027e70e07
I20260809 02:19:45.668941 8605 leader_election.cc:302] T 1a3b6ba630cc434d8898d43027e70e07 P 8503d46e570848f0af6333917d9fee07 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8503d46e570848f0af6333917d9fee07; no voters: 29341fbfcc86453da034b2c10e418b3e, 4ba2d99b0bd74c71b736bed8f281e453
I20260809 02:19:45.669093 9236 raft_consensus.cc:2726] T 1a3b6ba630cc434d8898d43027e70e07 P 8503d46e570848f0af6333917d9fee07 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 02:19:45.800091 9237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:45.799659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36755 (local address 127.15.75.131:36571)
0809 02:19:45.799937 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:45.799994 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:36755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":72,"threads_started":1}
I20260809 02:19:46.467743 9238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:46.467342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36755 (local address 127.15.75.129:47907)
0809 02:19:46.467585 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:46.467640 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:36755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":75,"threads_started":1}
I20260809 02:19:46.618619 9239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:46.618225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36755 (local address 127.15.75.130:60999)
0809 02:19:46.618456 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:46.618508 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:36755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":75,"threads_started":1}
I20260809 02:19:46.801328 9240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:46.800946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36755 (local address 127.15.75.131:54187)
0809 02:19:46.801187 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:46.801240 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:36755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":73,"threads_started":1}
I20260809 02:19:47.469059 9241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:47.468557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36755 (local address 127.15.75.129:56965)
0809 02:19:47.468907 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:47.468962 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:36755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":150,"threads_started":1}
I20260809 02:19:47.619849 9242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:47.619447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36755 (local address 127.15.75.130:58203)
0809 02:19:47.619703 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:47.619758 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:36755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":91,"threads_started":1}
I20260809 02:19:47.802412 9244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:47.802029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36755 (local address 127.15.75.131:45439)
0809 02:19:47.802263 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:47.802315 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:36755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":77,"threads_started":1}
I20260809 02:19:48.470317 9245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:48.469830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36755 (local address 127.15.75.129:56221)
0809 02:19:48.470162 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:48.470219 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:36755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":79,"threads_started":1}
I20260809 02:19:48.621096 9246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:48.620598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36755 (local address 127.15.75.130:52427)
0809 02:19:48.620944 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:48.621001 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:36755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":171,"threads_started":1}
I20260809 02:19:48.803506 9247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:48.803104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36755 (local address 127.15.75.131:43169)
0809 02:19:48.803358 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:48.803415 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:36755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":75,"threads_started":1}
I20260809 02:19:49.471433 9249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:49.471016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36755 (local address 127.15.75.129:49253)
0809 02:19:49.471268 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:49.471333 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:36755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":75,"threads_started":1}
I20260809 02:19:49.622229 9250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:49.621830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36755 (local address 127.15.75.130:39693)
0809 02:19:49.622083 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:49.622137 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:36755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":84,"threads_started":1}
I20260809 02:19:49.804661 9252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:49.804267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36755 (local address 127.15.75.131:40739)
0809 02:19:49.804521 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:49.804572 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:36755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":96,"threads_started":1}
I20260809 02:19:50.443300 9254 raft_consensus.cc:482] T 1a3b6ba630cc434d8898d43027e70e07 P 8503d46e570848f0af6333917d9fee07 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:19:50.443492 9254 raft_consensus.cc:504] T 1a3b6ba630cc434d8898d43027e70e07 P 8503d46e570848f0af6333917d9fee07 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29341fbfcc86453da034b2c10e418b3e" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45381 } } peers { permanent_uuid: "4ba2d99b0bd74c71b736bed8f281e453" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 45603 } } peers { permanent_uuid: "8503d46e570848f0af6333917d9fee07" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 40471 } }
I20260809 02:19:50.443959 9254 leader_election.cc:288] T 1a3b6ba630cc434d8898d43027e70e07 P 8503d46e570848f0af6333917d9fee07 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 29341fbfcc86453da034b2c10e418b3e (127.15.75.130:45381), 4ba2d99b0bd74c71b736bed8f281e453 (127.15.75.131:45603)
W20260809 02:19:50.444765 8603 leader_election.cc:341] T 1a3b6ba630cc434d8898d43027e70e07 P 8503d46e570848f0af6333917d9fee07 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 29341fbfcc86453da034b2c10e418b3e (127.15.75.130:45381): Not found: Tablet not found: 1a3b6ba630cc434d8898d43027e70e07
W20260809 02:19:50.444947 8605 leader_election.cc:341] T 1a3b6ba630cc434d8898d43027e70e07 P 8503d46e570848f0af6333917d9fee07 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4ba2d99b0bd74c71b736bed8f281e453 (127.15.75.131:45603): Not found: Tablet not found: 1a3b6ba630cc434d8898d43027e70e07
I20260809 02:19:50.445022 8605 leader_election.cc:302] T 1a3b6ba630cc434d8898d43027e70e07 P 8503d46e570848f0af6333917d9fee07 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8503d46e570848f0af6333917d9fee07; no voters: 29341fbfcc86453da034b2c10e418b3e, 4ba2d99b0bd74c71b736bed8f281e453
I20260809 02:19:50.445151 9254 raft_consensus.cc:2726] T 1a3b6ba630cc434d8898d43027e70e07 P 8503d46e570848f0af6333917d9fee07 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 02:19:50.472844 9255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:50.472377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36755 (local address 127.15.75.129:44303)
0809 02:19:50.472640 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:50.472750 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:36755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":89,"threads_started":1}
I20260809 02:19:50.623454 9256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:50.623064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36755 (local address 127.15.75.130:46643)
0809 02:19:50.623317 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:50.623368 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:36755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":89,"threads_started":1}
I20260809 02:19:50.796830 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 8499
I20260809 02:19:50.805963 9257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:50.805564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:36755 (local address 127.15.75.131:40741)
0809 02:19:50.805801 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:50.805872 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.190:36755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":136,"threads_started":1}
I20260809 02:19:50.810381 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 8793
I20260809 02:19:50.820946 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 8983
2026-08-09T02:19:50Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59355 ms)
[----------] 30 tests from SecurityITest (249028 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:19:50 dist-test-slave-s11t krb5kdc[9260](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:19:50 dist-test-slave-s11t krb5kdc[9260](info): set up 2 sockets
Aug 09 02:19:50 dist-test-slave-s11t krb5kdc[9260](info): commencing operation
krb5kdc: starting...
W20260809 02:19:52.064181 15662 mini_kdc.cc:121] Time spent starting KDC: real 1.217s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Aug 09 02:19:52 dist-test-slave-s11t krb5kdc[9260](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241992, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:19:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:19:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.75.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab.
Entry for principal kudu/127.15.75.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab.
I20260809 02:19:52.185753 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:41203
--webserver_interface=127.15.75.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab
--principal=kudu/127.15.75.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:35555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.75.190:41203
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:19:52.298862 9275 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:19:52.299096 9275 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:19:52.299134 9275 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:19:52.302956 9275 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:19:52.303005 9275 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:19:52.303025 9275 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:19:52.303040 9275 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:19:52.303054 9275 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:19:52.308007 9275 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:35555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.75.190:41203
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab
--principal=kudu/127.15.75.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:41203
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.75.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9275
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:19:52.308921 9275 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:19:52.309867 9275 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:19:52.318614 9286 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:19:52.318643 9287 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:19:52.318605 9290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:19:52.318904 9275 server_base.cc:1034] running on GCE node
I20260809 02:19:52.319429 9275 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:19:52.320399 9275 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:19:52.321563 9275 hybrid_clock.cc:648] HybridClock initialized: now 1786241992321523 us; error 51 us; skew 500 ppm
Aug 09 02:19:52 dist-test-slave-s11t krb5kdc[9260](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241992, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:19:52.324273 9275 init.cc:377] Logged in from keytab as kudu/127.15.75.190@KRBTEST.COM (short username kudu)
I20260809 02:19:52.325184 9275 webserver.cc:460] Webserver started at http://127.15.75.190:33091/ using document root <none> and password file <none>
I20260809 02:19:52.325711 9275 fs_manager.cc:359] Metadata directory not provided
I20260809 02:19:52.325771 9275 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:19:52.326027 9275 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:19:52.327962 9275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/minicluster-data/master-0/data/instance:
uuid: "7e9165c554dc4389984130d40adcfc31"
format_stamp: "Formatted at 2026-08-09 02:19:52 on dist-test-slave-s11t"
server_key: "4e088725cb9d48aaf8da43e1a6b3cf21"
server_key_iv: "b99157120f124700422f2723252bcba5"
server_key_version: "encryptionkey@0"
I20260809 02:19:52.328508 9275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/minicluster-data/master-0/wal/instance:
uuid: "7e9165c554dc4389984130d40adcfc31"
format_stamp: "Formatted at 2026-08-09 02:19:52 on dist-test-slave-s11t"
server_key: "4e088725cb9d48aaf8da43e1a6b3cf21"
server_key_iv: "b99157120f124700422f2723252bcba5"
server_key_version: "encryptionkey@0"
I20260809 02:19:52.332031 9275 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260809 02:19:52.334412 9302 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:19:52.335677 9275 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260809 02:19:52.335814 9275 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/minicluster-data/master-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/minicluster-data/master-0/wal
uuid: "7e9165c554dc4389984130d40adcfc31"
format_stamp: "Formatted at 2026-08-09 02:19:52 on dist-test-slave-s11t"
server_key: "4e088725cb9d48aaf8da43e1a6b3cf21"
server_key_iv: "b99157120f124700422f2723252bcba5"
server_key_version: "encryptionkey@0"
I20260809 02:19:52.335942 9275 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:19:52.348106 9275 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:19:52.350898 9275 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:19:52.351082 9275 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:19:52.358667 9275 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.190:41203
I20260809 02:19:52.358693 9378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.190:41203 every 8 connection(s)
I20260809 02:19:52.359560 9275 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
I20260809 02:19:52.361832 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 9275
I20260809 02:19:52.361971 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/minicluster-data/master-0/wal/instance
I20260809 02:19:52.362241 15662 external_mini_cluster.cc:1428] Setting key 6422ad0fe1b76280d2f069cb8c99e50b
I20260809 02:19:52.362527 9380 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:19:52.369110 9380 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e9165c554dc4389984130d40adcfc31: Bootstrap starting.
Aug 09 02:19:52 dist-test-slave-s11t krb5kdc[9260](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241992, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:19:52.371934 9380 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7e9165c554dc4389984130d40adcfc31: Neither blocks nor log segments found. Creating new log.
I20260809 02:19:52.372886 9380 log.cc:824] T 00000000000000000000000000000000 P 7e9165c554dc4389984130d40adcfc31: Log is configured to *not* fsync() on all Append() calls
I20260809 02:19:52.375321 9380 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e9165c554dc4389984130d40adcfc31: No bootstrap required, opened a new log
I20260809 02:19:52.376489 9383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:52.363643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59242 (local address 127.15.75.190:41203)
0809 02:19:52.364101 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:52.364115 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:52.364150 (+ 35us) server_negotiation.cc:407] Connection header received
0809 02:19:52.364850 (+ 700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:52.364867 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:52.365169 (+ 302us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:52.365485 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:52.366264 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:52.367243 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:52.368103 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:52.368559 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:52.368776 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:52.371612 (+ 2836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:19:52.371617 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:19:52.371629 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:19:52.371655 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:19:52.373628 (+ 1973us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:52.374141 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:52.374146 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:52.374150 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:52.374254 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:52.374567 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:52.374570 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:52.374571 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:52.374923 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:19:52.375061 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:52.375504 (+ 443us) server_negotiation.cc:299] Negotiation successful
0809 02:19:52.375812 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":165,"threads_started":1}
I20260809 02:19:52.378659 9380 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7e9165c554dc4389984130d40adcfc31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e9165c554dc4389984130d40adcfc31" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 41203 } }
I20260809 02:19:52.378921 9380 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7e9165c554dc4389984130d40adcfc31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:19:52.379006 9380 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7e9165c554dc4389984130d40adcfc31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e9165c554dc4389984130d40adcfc31, State: Initialized, Role: FOLLOWER
I20260809 02:19:52.379522 9380 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e9165c554dc4389984130d40adcfc31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e9165c554dc4389984130d40adcfc31" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 41203 } }
I20260809 02:19:52.379657 9380 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7e9165c554dc4389984130d40adcfc31 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:19:52.379738 9380 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7e9165c554dc4389984130d40adcfc31 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:19:52.379845 9380 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e9165c554dc4389984130d40adcfc31 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:19:52.380805 9380 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7e9165c554dc4389984130d40adcfc31 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e9165c554dc4389984130d40adcfc31" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 41203 } }
I20260809 02:19:52.381150 9380 leader_election.cc:302] T 00000000000000000000000000000000 P 7e9165c554dc4389984130d40adcfc31 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7e9165c554dc4389984130d40adcfc31; no voters:
I20260809 02:19:52.381496 9380 leader_election.cc:288] T 00000000000000000000000000000000 P 7e9165c554dc4389984130d40adcfc31 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:19:52.381639 9385 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7e9165c554dc4389984130d40adcfc31 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:19:52.381884 9385 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7e9165c554dc4389984130d40adcfc31 [term 1 LEADER]: Becoming Leader. State: Replica: 7e9165c554dc4389984130d40adcfc31, State: Running, Role: LEADER
I20260809 02:19:52.382254 9385 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7e9165c554dc4389984130d40adcfc31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e9165c554dc4389984130d40adcfc31" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 41203 } }
I20260809 02:19:52.382707 9380 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7e9165c554dc4389984130d40adcfc31 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:19:52.383906 9386 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e9165c554dc4389984130d40adcfc31 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7e9165c554dc4389984130d40adcfc31" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e9165c554dc4389984130d40adcfc31" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 41203 } } }
I20260809 02:19:52.384045 9386 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e9165c554dc4389984130d40adcfc31 [sys.catalog]: This master's current role is: LEADER
I20260809 02:19:52.384398 9387 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e9165c554dc4389984130d40adcfc31 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7e9165c554dc4389984130d40adcfc31. Latest consensus state: current_term: 1 leader_uuid: "7e9165c554dc4389984130d40adcfc31" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e9165c554dc4389984130d40adcfc31" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 41203 } } }
I20260809 02:19:52.384557 9387 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e9165c554dc4389984130d40adcfc31 [sys.catalog]: This master's current role is: LEADER
I20260809 02:19:52.384616 9395 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:19:52.387787 9395 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:19:52.393628 9395 catalog_manager.cc:1347] Generated new cluster ID: 947d15fc040f4d3a9ed3aaf737384985
I20260809 02:19:52.393703 9395 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:19:52.420275 9395 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:19:52.421548 9395 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:19:52.427286 9395 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7e9165c554dc4389984130d40adcfc31: Generated new TSK 0
I20260809 02:19:52.427955 9395 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.75.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab.
Entry for principal kudu/127.15.75.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab.
I20260809 02:19:52.463847 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.129:0
--local_ip_for_outbound_sockets=127.15.75.129
--webserver_interface=127.15.75.129
--webserver_port=0
--tserver_master_addrs=127.15.75.190:41203
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab
--principal=kudu/127.15.75.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:35555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:19:52.574891 9416 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:19:52.575115 9416 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:19:52.575155 9416 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:19:52.579123 9416 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:19:52.579171 9416 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:19:52.579249 9416 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.129
I20260809 02:19:52.584547 9416 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:35555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab
--principal=kudu/127.15.75.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.75.129
--webserver_port=0
--tserver_master_addrs=127.15.75.190:41203
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9416
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.129
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:19:52.585489 9416 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:19:52.586470 9416 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:19:52.595779 9426 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:19:52.595912 9416 server_base.cc:1034] running on GCE node
W20260809 02:19:52.595978 9429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:19:52.595880 9425 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:19:52.596478 9416 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:19:52.597117 9416 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:19:52.598279 9416 hybrid_clock.cc:648] HybridClock initialized: now 1786241992598246 us; error 45 us; skew 500 ppm
Aug 09 02:19:52 dist-test-slave-s11t krb5kdc[9260](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241992, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:19:52.601238 9416 init.cc:377] Logged in from keytab as kudu/127.15.75.129@KRBTEST.COM (short username kudu)
I20260809 02:19:52.602056 9416 webserver.cc:460] Webserver started at http://127.15.75.129:45331/ using document root <none> and password file <none>
I20260809 02:19:52.602594 9416 fs_manager.cc:359] Metadata directory not provided
I20260809 02:19:52.602655 9416 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:19:52.602873 9416 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:19:52.604871 9416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/minicluster-data/ts-0/data/instance:
uuid: "4a28c730071b43f2a39b62fcf3915d51"
format_stamp: "Formatted at 2026-08-09 02:19:52 on dist-test-slave-s11t"
server_key: "798746f680632cdf1ae52a7fa06f8e7a"
server_key_iv: "a1ff3f9e3090f4d7ee69b33dfec0abfb"
server_key_version: "encryptionkey@0"
I20260809 02:19:52.605429 9416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/minicluster-data/ts-0/wal/instance:
uuid: "4a28c730071b43f2a39b62fcf3915d51"
format_stamp: "Formatted at 2026-08-09 02:19:52 on dist-test-slave-s11t"
server_key: "798746f680632cdf1ae52a7fa06f8e7a"
server_key_iv: "a1ff3f9e3090f4d7ee69b33dfec0abfb"
server_key_version: "encryptionkey@0"
I20260809 02:19:52.608968 9416 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 02:19:52.611506 9440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:19:52.612818 9416 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 02:19:52.612950 9416 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/minicluster-data/ts-0/wal
uuid: "4a28c730071b43f2a39b62fcf3915d51"
format_stamp: "Formatted at 2026-08-09 02:19:52 on dist-test-slave-s11t"
server_key: "798746f680632cdf1ae52a7fa06f8e7a"
server_key_iv: "a1ff3f9e3090f4d7ee69b33dfec0abfb"
server_key_version: "encryptionkey@0"
I20260809 02:19:52.613057 9416 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:19:52.627741 9416 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:19:52.630543 9416 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:19:52.630738 9416 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:19:52.631301 9416 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:19:52.632113 9416 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:19:52.632182 9416 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:19:52.632254 9416 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:19:52.632295 9416 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:19:52.642349 9416 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.129:38761
I20260809 02:19:52.642390 9628 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.129:38761 every 8 connection(s)
I20260809 02:19:52.643226 9416 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
I20260809 02:19:52.650419 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 9416
I20260809 02:19:52.650555 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/minicluster-data/ts-0/wal/instance
I20260809 02:19:52.650902 15662 external_mini_cluster.cc:1428] Setting key 53ad6cdcaa4906f530cf00558a45a450
Aug 09 02:19:52 dist-test-slave-s11t krb5kdc[9260](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241992, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.129@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:19:52.657220 9383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:52.644976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.129:37717 (local address 127.15.75.190:41203)
0809 02:19:52.645101 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:52.645112 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:52.645876 (+ 764us) server_negotiation.cc:407] Connection header received
0809 02:19:52.647203 (+ 1327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:52.647206 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:52.647279 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:52.647353 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:52.648438 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:52.649075 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:52.650248 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:52.650525 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:52.650660 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:52.653369 (+ 2709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:19:52.653375 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:19:52.653382 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:19:52.653404 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:19:52.654965 (+ 1561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:52.655572 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:52.655574 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:52.655575 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:52.655651 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:52.656074 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:52.656077 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:52.656079 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:52.656257 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:19:52.656373 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:52.656924 (+ 551us) server_negotiation.cc:299] Negotiation successful
0809 02:19:52.657051 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 02:19:52.658054 9631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:52.645291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:41203 (local address 127.15.75.129:37717)
0809 02:19:52.645716 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:52.645757 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 02:19:52.646993 (+ 1236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:19:52.647497 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:19:52.647504 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:19:52.647829 (+ 325us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:19:52.648290 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:52.648304 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:52.649201 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:52.649203 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:52.650072 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:52.650081 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:52.650701 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:52.650706 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:52.650947 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:52.651529 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:19:52.651545 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:19:52.653209 (+ 1664us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:19:52.655085 (+ 1876us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:19:52.655092 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:19:52.655109 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:19:52.655472 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:19:52.655745 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:19:52.655750 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:19:52.655755 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:19:52.655948 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:19:52.656387 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:19:52.656395 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:19:52.656625 (+ 230us) client_negotiation.cc:769] Sending connection context
0809 02:19:52.656861 (+ 236us) client_negotiation.cc:240] Negotiation successful
0809 02:19:52.657169 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":112,"threads_started":1}
I20260809 02:19:52.659330 9629 heartbeater.cc:347] Connected to a master server at 127.15.75.190:41203
I20260809 02:19:52.659574 9629 heartbeater.cc:464] Registering TS with master...
I20260809 02:19:52.660041 9629 heartbeater.cc:511] Master 127.15.75.190:41203 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.15.75.130@KRBTEST.COM; defaulting to no policy
I20260809 02:19:52.661768 9330 ts_manager.cc:194] Registered new tserver with Master: 4a28c730071b43f2a39b62fcf3915d51 (127.15.75.129:38761)
I20260809 02:19:52.663005 9330 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.75.129@KRBTEST.COM'} at 127.15.75.129:37717
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab.
Entry for principal kudu/127.15.75.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab.
I20260809 02:19:52.679294 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.130:0
--local_ip_for_outbound_sockets=127.15.75.130
--webserver_interface=127.15.75.130
--webserver_port=0
--tserver_master_addrs=127.15.75.190:41203
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab
--principal=kudu/127.15.75.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:35555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:19:52.788796 9635 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:19:52.789047 9635 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:19:52.789089 9635 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:19:52.793071 9635 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:19:52.793121 9635 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:19:52.793179 9635 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.130
I20260809 02:19:52.798388 9635 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:35555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab
--principal=kudu/127.15.75.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.75.130
--webserver_port=0
--tserver_master_addrs=127.15.75.190:41203
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9635
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.130
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:19:52.799345 9635 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:19:52.800314 9635 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:19:52.810250 9644 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:19:52.810168 9640 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:19:52.810171 9641 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:19:52.810525 9635 server_base.cc:1034] running on GCE node
I20260809 02:19:52.810912 9635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:19:52.811476 9635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:19:52.812767 9635 hybrid_clock.cc:648] HybridClock initialized: now 1786241992812720 us; error 47 us; skew 500 ppm
Aug 09 02:19:52 dist-test-slave-s11t krb5kdc[9260](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241992, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:19:52.815572 9635 init.cc:377] Logged in from keytab as kudu/127.15.75.130@KRBTEST.COM (short username kudu)
I20260809 02:19:52.816636 9635 webserver.cc:460] Webserver started at http://127.15.75.130:45117/ using document root <none> and password file <none>
I20260809 02:19:52.817324 9635 fs_manager.cc:359] Metadata directory not provided
I20260809 02:19:52.817417 9635 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:19:52.817634 9635 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:19:52.819653 9635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/minicluster-data/ts-1/data/instance:
uuid: "b379803659d2405f8826b28a044dde77"
format_stamp: "Formatted at 2026-08-09 02:19:52 on dist-test-slave-s11t"
server_key: "e3f63296fedf3d08e61381b963bafbb7"
server_key_iv: "09153b8817bf7d1e676787f97ac1f771"
server_key_version: "encryptionkey@0"
I20260809 02:19:52.820231 9635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/minicluster-data/ts-1/wal/instance:
uuid: "b379803659d2405f8826b28a044dde77"
format_stamp: "Formatted at 2026-08-09 02:19:52 on dist-test-slave-s11t"
server_key: "e3f63296fedf3d08e61381b963bafbb7"
server_key_iv: "09153b8817bf7d1e676787f97ac1f771"
server_key_version: "encryptionkey@0"
I20260809 02:19:52.823925 9635 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260809 02:19:52.826591 9653 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:19:52.827828 9635 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 02:19:52.827982 9635 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/minicluster-data/ts-1/wal
uuid: "b379803659d2405f8826b28a044dde77"
format_stamp: "Formatted at 2026-08-09 02:19:52 on dist-test-slave-s11t"
server_key: "e3f63296fedf3d08e61381b963bafbb7"
server_key_iv: "09153b8817bf7d1e676787f97ac1f771"
server_key_version: "encryptionkey@0"
I20260809 02:19:52.828096 9635 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:19:52.842374 9635 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:19:52.845777 9635 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:19:52.846009 9635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:19:52.846613 9635 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:19:52.847460 9635 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:19:52.847533 9635 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:19:52.847605 9635 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:19:52.847658 9635 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:19:52.856854 9635 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.130:42105
I20260809 02:19:52.856886 9831 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.130:42105 every 8 connection(s)
I20260809 02:19:52.857717 9635 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
I20260809 02:19:52.865568 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 9635
I20260809 02:19:52.865741 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/minicluster-data/ts-1/wal/instance
I20260809 02:19:52.866063 15662 external_mini_cluster.cc:1428] Setting key c9dc18bcd4f51722cc39ab934990d19d
Aug 09 02:19:52 dist-test-slave-s11t krb5kdc[9260](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241992, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.130@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:19:52.872378 9383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:52.859495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.130:56351 (local address 127.15.75.190:41203)
0809 02:19:52.859642 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:52.859646 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:52.860454 (+ 808us) server_negotiation.cc:407] Connection header received
0809 02:19:52.861847 (+ 1393us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:52.861850 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:52.861908 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:52.862019 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:52.863166 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:52.863712 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:52.864937 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:52.865167 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:52.865286 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:52.868244 (+ 2958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:19:52.868250 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:19:52.868256 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:19:52.868290 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:19:52.870165 (+ 1875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:52.870829 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:52.870832 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:52.870833 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:52.870911 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:52.871300 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:52.871302 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:52.871303 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:52.871469 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:19:52.871544 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:52.872156 (+ 612us) server_negotiation.cc:299] Negotiation successful
0809 02:19:52.872253 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 02:19:52.873646 9835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:52.859865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:41203 (local address 127.15.75.130:56351)
0809 02:19:52.860303 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:52.860345 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 02:19:52.861637 (+ 1292us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:19:52.862194 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:19:52.862201 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:19:52.862527 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:19:52.863012 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:52.863025 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:52.863848 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:52.863852 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:52.864806 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:52.864816 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:52.865323 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:52.865327 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:52.865503 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:52.866180 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:19:52.866205 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:19:52.868060 (+ 1855us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:19:52.870304 (+ 2244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:19:52.870313 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:19:52.870328 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:19:52.870717 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:19:52.871004 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:19:52.871009 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:19:52.871012 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:19:52.871199 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:19:52.871575 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:19:52.871583 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:19:52.871907 (+ 324us) client_negotiation.cc:769] Sending connection context
0809 02:19:52.872118 (+ 211us) client_negotiation.cc:240] Negotiation successful
0809 02:19:52.872468 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":115,"threads_started":1}
I20260809 02:19:52.874894 9832 heartbeater.cc:347] Connected to a master server at 127.15.75.190:41203
I20260809 02:19:52.875137 9832 heartbeater.cc:464] Registering TS with master...
I20260809 02:19:52.875842 9832 heartbeater.cc:511] Master 127.15.75.190:41203 requested a full tablet report, sending...
I20260809 02:19:52.876971 9330 ts_manager.cc:194] Registered new tserver with Master: b379803659d2405f8826b28a044dde77 (127.15.75.130:42105)
I20260809 02:19:52.877545 9330 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.75.130@KRBTEST.COM'} at 127.15.75.130:56351
WARNING: no policy specified for kudu/127.15.75.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab.
Entry for principal kudu/127.15.75.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab.
I20260809 02:19:52.895545 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.131:0
--local_ip_for_outbound_sockets=127.15.75.131
--webserver_interface=127.15.75.131
--webserver_port=0
--tserver_master_addrs=127.15.75.190:41203
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab
--principal=kudu/127.15.75.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:35555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:19:53.007831 9839 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:19:53.008077 9839 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:19:53.008118 9839 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:19:53.012244 9839 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:19:53.012300 9839 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:19:53.012387 9839 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.131
I20260809 02:19:53.018502 9839 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:35555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab
--principal=kudu/127.15.75.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.75.131
--webserver_port=0
--tserver_master_addrs=127.15.75.190:41203
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9839
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.131
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:19:53.019558 9839 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:19:53.020637 9839 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:19:53.030789 9851 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:19:53.030825 9856 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:19:53.030800 9852 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:19:53.031210 9839 server_base.cc:1034] running on GCE node
I20260809 02:19:53.031663 9839 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:19:53.032317 9839 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:19:53.033492 9839 hybrid_clock.cc:648] HybridClock initialized: now 1786241993033455 us; error 46 us; skew 500 ppm
Aug 09 02:19:53 dist-test-slave-s11t krb5kdc[9260](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241993, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:19:53.036475 9839 init.cc:377] Logged in from keytab as kudu/127.15.75.131@KRBTEST.COM (short username kudu)
I20260809 02:19:53.037310 9839 webserver.cc:460] Webserver started at http://127.15.75.131:46269/ using document root <none> and password file <none>
I20260809 02:19:53.037799 9839 fs_manager.cc:359] Metadata directory not provided
I20260809 02:19:53.037864 9839 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:19:53.038023 9839 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:19:53.040022 9839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/minicluster-data/ts-2/data/instance:
uuid: "629a4463a87648bfabc61517716e9062"
format_stamp: "Formatted at 2026-08-09 02:19:53 on dist-test-slave-s11t"
server_key: "2b490b9e7502e95b9edd2e350da94cf3"
server_key_iv: "97d72ed0147b090dcf8c955f031404a6"
server_key_version: "encryptionkey@0"
I20260809 02:19:53.040582 9839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/minicluster-data/ts-2/wal/instance:
uuid: "629a4463a87648bfabc61517716e9062"
format_stamp: "Formatted at 2026-08-09 02:19:53 on dist-test-slave-s11t"
server_key: "2b490b9e7502e95b9edd2e350da94cf3"
server_key_iv: "97d72ed0147b090dcf8c955f031404a6"
server_key_version: "encryptionkey@0"
I20260809 02:19:53.044171 9839 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260809 02:19:53.046643 9865 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:19:53.047971 9839 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 02:19:53.048115 9839 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/minicluster-data/ts-2/wal
uuid: "629a4463a87648bfabc61517716e9062"
format_stamp: "Formatted at 2026-08-09 02:19:53 on dist-test-slave-s11t"
server_key: "2b490b9e7502e95b9edd2e350da94cf3"
server_key_iv: "97d72ed0147b090dcf8c955f031404a6"
server_key_version: "encryptionkey@0"
I20260809 02:19:53.048228 9839 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:19:53.064836 9839 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:19:53.068023 9839 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:19:53.068223 9839 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:19:53.068812 9839 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:19:53.069967 9839 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:19:53.070052 9839 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:19:53.070127 9839 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:19:53.070179 9839 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:19:53.079950 9839 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.131:33115
I20260809 02:19:53.079973 10044 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.131:33115 every 8 connection(s)
I20260809 02:19:53.080948 9839 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
I20260809 02:19:53.081812 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 9839
I20260809 02:19:53.081984 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/minicluster-data/ts-2/wal/instance
I20260809 02:19:53.082392 15662 external_mini_cluster.cc:1428] Setting key 016321b45f28c371b4f7041f278366d9
Aug 09 02:19:53 dist-test-slave-s11t krb5kdc[9260](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241993, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.131@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:19:53.095865 9383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:53.083032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.131:55385 (local address 127.15.75.190:41203)
0809 02:19:53.083217 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:53.083223 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:53.084251 (+ 1028us) server_negotiation.cc:407] Connection header received
0809 02:19:53.085910 (+ 1659us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:53.085913 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:53.085972 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:53.086058 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:53.087276 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:53.087876 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:53.089236 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:53.089500 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:53.089624 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:53.092291 (+ 2667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:19:53.092295 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:19:53.092298 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:19:53.092326 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:19:53.093746 (+ 1420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:53.094254 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:53.094257 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:53.094259 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:53.094344 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:53.094683 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:53.094688 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:53.094689 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:53.094875 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:19:53.095003 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:53.095525 (+ 522us) server_negotiation.cc:299] Negotiation successful
0809 02:19:53.095668 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 02:19:53.096798 10047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:53.083476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:41203 (local address 127.15.75.131:55385)
0809 02:19:53.084106 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:53.084152 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 02:19:53.085683 (+ 1531us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:19:53.086236 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:19:53.086243 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:19:53.086609 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:19:53.087121 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:53.087136 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:53.088018 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:53.088022 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:53.089092 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:53.089103 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:53.089668 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:53.089672 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:53.089860 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:53.090527 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:19:53.090545 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:19:53.092111 (+ 1566us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:19:53.093869 (+ 1758us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:19:53.093876 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:19:53.093889 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:19:53.094144 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:19:53.094440 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:19:53.094445 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:19:53.094447 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:19:53.094584 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:19:53.094993 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:19:53.095000 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:19:53.095239 (+ 239us) client_negotiation.cc:769] Sending connection context
0809 02:19:53.095437 (+ 198us) client_negotiation.cc:240] Negotiation successful
0809 02:19:53.095798 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":132,"threads_started":1}
I20260809 02:19:53.097843 10045 heartbeater.cc:347] Connected to a master server at 127.15.75.190:41203
I20260809 02:19:53.098107 10045 heartbeater.cc:464] Registering TS with master...
I20260809 02:19:53.098584 10045 heartbeater.cc:511] Master 127.15.75.190:41203 requested a full tablet report, sending...
I20260809 02:19:53.099602 9330 ts_manager.cc:194] Registered new tserver with Master: 629a4463a87648bfabc61517716e9062 (127.15.75.131:33115)
I20260809 02:19:53.100178 9330 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.75.131@KRBTEST.COM'} at 127.15.75.131:55385
I20260809 02:19:53.108475 15662 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 02:19:53.117862 9383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:53.110491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59256 (local address 127.15.75.190:41203)
0809 02:19:53.110643 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:53.110650 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:53.110763 (+ 113us) server_negotiation.cc:407] Connection header received
0809 02:19:53.110859 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:53.110861 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:53.110902 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:53.110970 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:53.111664 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:53.112172 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:53.113041 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:53.113246 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:53.113324 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:53.114563 (+ 1239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:19:53.114565 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:19:53.114567 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:19:53.114588 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:19:53.116081 (+ 1493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:53.116455 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:53.116458 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:53.116459 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:53.116533 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:53.117033 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:53.117036 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:53.117040 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:53.117285 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:19:53.117393 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:53.117617 (+ 224us) server_negotiation.cc:299] Negotiation successful
0809 02:19:53.117707 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 02:19:53.120512 9330 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59256:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:19:53.123553 9330 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:19:53.137905 10062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:53.131667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.130:42105 (local address 127.0.0.1:34244)
0809 02:19:53.132176 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:53.132226 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 02:19:53.132356 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:19:53.133445 (+ 1089us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:19:53.133452 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:19:53.133477 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:19:53.133614 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:53.133620 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:53.135064 (+ 1444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:53.135067 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:53.136404 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:53.136416 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:53.137347 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:53.137350 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:53.137478 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:53.137520 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 02:19:53.137631 (+ 111us) client_negotiation.cc:240] Negotiation successful
0809 02:19:53.137751 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"mutex_wait_us":61,"thread_start_us":74,"threads_started":1}
I20260809 02:19:53.138036 10069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:53.132445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41056 (local address 127.15.75.131:33115)
0809 02:19:53.133463 (+ 1018us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:53.133475 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:53.133497 (+ 22us) server_negotiation.cc:407] Connection header received
0809 02:19:53.133567 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:53.133572 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:53.133764 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:53.134219 (+ 455us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:19:53.134346 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:53.135343 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:53.136672 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:53.137488 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:53.137596 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:53.137732 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:53.137818 (+ 86us) server_negotiation.cc:299] Negotiation successful
0809 02:19:53.137893 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":912,"thread_start_us":162,"threads_started":1}
I20260809 02:19:53.138353 10064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:53.131828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34244 (local address 127.15.75.130:42105)
0809 02:19:53.132677 (+ 849us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:53.132724 (+ 47us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:53.132750 (+ 26us) server_negotiation.cc:407] Connection header received
0809 02:19:53.132834 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:53.132841 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:53.133065 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:53.133226 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:19:53.133746 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:53.134918 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:53.136545 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:53.137212 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:53.137343 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:53.137494 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:53.138113 (+ 619us) server_negotiation.cc:299] Negotiation successful
0809 02:19:53.138215 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":738,"thread_start_us":138,"threads_started":1}
I20260809 02:19:53.138055 10066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:53.131868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.131:33115 (local address 127.0.0.1:41056)
0809 02:19:53.132607 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:53.132620 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:19:53.132727 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:19:53.133940 (+ 1213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:19:53.133943 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:19:53.133967 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:19:53.134070 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:53.134076 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:53.135528 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:53.135531 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:53.136540 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:53.136551 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:53.137596 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:53.137602 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:53.137701 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:53.137721 (+ 20us) client_negotiation.cc:769] Sending connection context
0809 02:19:53.137788 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 02:19:53.137842 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":676,"thread_start_us":105,"threads_started":1}
I20260809 02:19:53.138345 10067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:53.132399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.129:38761 (local address 127.0.0.1:34098)
0809 02:19:53.133130 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:53.133146 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 02:19:53.133220 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:19:53.133796 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:19:53.133799 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:19:53.133819 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:19:53.133940 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:53.133949 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:53.134924 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:53.134930 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:53.136445 (+ 1515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:53.136456 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:53.137959 (+ 1503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:53.137962 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:53.138080 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:53.138103 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 02:19:53.138174 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 02:19:53.138232 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":669,"mutex_wait_us":2,"thread_start_us":102,"threads_started":1}
I20260809 02:19:53.139029 10072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:53.132939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34098 (local address 127.15.75.129:38761)
0809 02:19:53.133366 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:53.133380 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:53.133398 (+ 18us) server_negotiation.cc:407] Connection header received
0809 02:19:53.133461 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:53.133465 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:53.133601 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:53.133813 (+ 212us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:19:53.134072 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:53.134793 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:53.136587 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:53.137479 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:53.137596 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:53.137730 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:53.138450 (+ 720us) server_negotiation.cc:299] Negotiation successful
0809 02:19:53.138558 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":122,"threads_started":1}
I20260809 02:19:53.141849 9942 tablet_service.cc:1467] Processing CreateTablet for tablet ba256336804d425ab1b4f5e8aa1f2b45 (DEFAULT_TABLE table=test-table [id=fe12f57cc29f45e3af04e5c74e625c0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:19:53.141839 9522 tablet_service.cc:1467] Processing CreateTablet for tablet ba256336804d425ab1b4f5e8aa1f2b45 (DEFAULT_TABLE table=test-table [id=fe12f57cc29f45e3af04e5c74e625c0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:19:53.142035 9733 tablet_service.cc:1467] Processing CreateTablet for tablet ba256336804d425ab1b4f5e8aa1f2b45 (DEFAULT_TABLE table=test-table [id=fe12f57cc29f45e3af04e5c74e625c0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:19:53.142966 9733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba256336804d425ab1b4f5e8aa1f2b45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:19:53.142993 9522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba256336804d425ab1b4f5e8aa1f2b45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:19:53.142993 9942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba256336804d425ab1b4f5e8aa1f2b45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:19:53.149147 10074 tablet_bootstrap.cc:492] T ba256336804d425ab1b4f5e8aa1f2b45 P b379803659d2405f8826b28a044dde77: Bootstrap starting.
I20260809 02:19:53.150417 10075 tablet_bootstrap.cc:492] T ba256336804d425ab1b4f5e8aa1f2b45 P 629a4463a87648bfabc61517716e9062: Bootstrap starting.
I20260809 02:19:53.151779 10074 tablet_bootstrap.cc:654] T ba256336804d425ab1b4f5e8aa1f2b45 P b379803659d2405f8826b28a044dde77: Neither blocks nor log segments found. Creating new log.
I20260809 02:19:53.152006 10076 tablet_bootstrap.cc:492] T ba256336804d425ab1b4f5e8aa1f2b45 P 4a28c730071b43f2a39b62fcf3915d51: Bootstrap starting.
I20260809 02:19:53.153071 10074 log.cc:824] T ba256336804d425ab1b4f5e8aa1f2b45 P b379803659d2405f8826b28a044dde77: Log is configured to *not* fsync() on all Append() calls
I20260809 02:19:53.153512 10075 tablet_bootstrap.cc:654] T ba256336804d425ab1b4f5e8aa1f2b45 P 629a4463a87648bfabc61517716e9062: Neither blocks nor log segments found. Creating new log.
I20260809 02:19:53.154196 10076 tablet_bootstrap.cc:654] T ba256336804d425ab1b4f5e8aa1f2b45 P 4a28c730071b43f2a39b62fcf3915d51: Neither blocks nor log segments found. Creating new log.
I20260809 02:19:53.154842 10075 log.cc:824] T ba256336804d425ab1b4f5e8aa1f2b45 P 629a4463a87648bfabc61517716e9062: Log is configured to *not* fsync() on all Append() calls
I20260809 02:19:53.154922 10074 tablet_bootstrap.cc:492] T ba256336804d425ab1b4f5e8aa1f2b45 P b379803659d2405f8826b28a044dde77: No bootstrap required, opened a new log
I20260809 02:19:53.155121 10074 ts_tablet_manager.cc:1397] T ba256336804d425ab1b4f5e8aa1f2b45 P b379803659d2405f8826b28a044dde77: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 02:19:53.155433 10076 log.cc:824] T ba256336804d425ab1b4f5e8aa1f2b45 P 4a28c730071b43f2a39b62fcf3915d51: Log is configured to *not* fsync() on all Append() calls
I20260809 02:19:53.157095 10076 tablet_bootstrap.cc:492] T ba256336804d425ab1b4f5e8aa1f2b45 P 4a28c730071b43f2a39b62fcf3915d51: No bootstrap required, opened a new log
I20260809 02:19:53.157305 10076 ts_tablet_manager.cc:1397] T ba256336804d425ab1b4f5e8aa1f2b45 P 4a28c730071b43f2a39b62fcf3915d51: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.004s
I20260809 02:19:53.158334 10074 raft_consensus.cc:348] T ba256336804d425ab1b4f5e8aa1f2b45 P b379803659d2405f8826b28a044dde77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b379803659d2405f8826b28a044dde77" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 42105 } } peers { permanent_uuid: "629a4463a87648bfabc61517716e9062" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 33115 } } peers { permanent_uuid: "4a28c730071b43f2a39b62fcf3915d51" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 38761 } }
I20260809 02:19:53.158535 10074 raft_consensus.cc:374] T ba256336804d425ab1b4f5e8aa1f2b45 P b379803659d2405f8826b28a044dde77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:19:53.158593 10074 raft_consensus.cc:729] T ba256336804d425ab1b4f5e8aa1f2b45 P b379803659d2405f8826b28a044dde77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b379803659d2405f8826b28a044dde77, State: Initialized, Role: FOLLOWER
I20260809 02:19:53.159042 10074 consensus_queue.cc:260] T ba256336804d425ab1b4f5e8aa1f2b45 P b379803659d2405f8826b28a044dde77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b379803659d2405f8826b28a044dde77" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 42105 } } peers { permanent_uuid: "629a4463a87648bfabc61517716e9062" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 33115 } } peers { permanent_uuid: "4a28c730071b43f2a39b62fcf3915d51" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 38761 } }
I20260809 02:19:53.159739 9832 heartbeater.cc:503] Master 127.15.75.190:41203 was elected leader, sending a full tablet report...
I20260809 02:19:53.160006 10074 ts_tablet_manager.cc:1428] T ba256336804d425ab1b4f5e8aa1f2b45 P b379803659d2405f8826b28a044dde77: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260809 02:19:53.160785 10076 raft_consensus.cc:348] T ba256336804d425ab1b4f5e8aa1f2b45 P 4a28c730071b43f2a39b62fcf3915d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b379803659d2405f8826b28a044dde77" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 42105 } } peers { permanent_uuid: "629a4463a87648bfabc61517716e9062" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 33115 } } peers { permanent_uuid: "4a28c730071b43f2a39b62fcf3915d51" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 38761 } }
I20260809 02:19:53.161062 10076 raft_consensus.cc:374] T ba256336804d425ab1b4f5e8aa1f2b45 P 4a28c730071b43f2a39b62fcf3915d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:19:53.161160 10076 raft_consensus.cc:729] T ba256336804d425ab1b4f5e8aa1f2b45 P 4a28c730071b43f2a39b62fcf3915d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a28c730071b43f2a39b62fcf3915d51, State: Initialized, Role: FOLLOWER
I20260809 02:19:53.161564 10076 consensus_queue.cc:260] T ba256336804d425ab1b4f5e8aa1f2b45 P 4a28c730071b43f2a39b62fcf3915d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b379803659d2405f8826b28a044dde77" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 42105 } } peers { permanent_uuid: "629a4463a87648bfabc61517716e9062" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 33115 } } peers { permanent_uuid: "4a28c730071b43f2a39b62fcf3915d51" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 38761 } }
I20260809 02:19:53.161770 10075 tablet_bootstrap.cc:492] T ba256336804d425ab1b4f5e8aa1f2b45 P 629a4463a87648bfabc61517716e9062: No bootstrap required, opened a new log
I20260809 02:19:53.161959 10075 ts_tablet_manager.cc:1397] T ba256336804d425ab1b4f5e8aa1f2b45 P 629a4463a87648bfabc61517716e9062: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.000s
I20260809 02:19:53.162175 9629 heartbeater.cc:503] Master 127.15.75.190:41203 was elected leader, sending a full tablet report...
I20260809 02:19:53.162513 10076 ts_tablet_manager.cc:1428] T ba256336804d425ab1b4f5e8aa1f2b45 P 4a28c730071b43f2a39b62fcf3915d51: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260809 02:19:53.165783 10075 raft_consensus.cc:348] T ba256336804d425ab1b4f5e8aa1f2b45 P 629a4463a87648bfabc61517716e9062 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b379803659d2405f8826b28a044dde77" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 42105 } } peers { permanent_uuid: "629a4463a87648bfabc61517716e9062" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 33115 } } peers { permanent_uuid: "4a28c730071b43f2a39b62fcf3915d51" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 38761 } }
I20260809 02:19:53.165966 10075 raft_consensus.cc:374] T ba256336804d425ab1b4f5e8aa1f2b45 P 629a4463a87648bfabc61517716e9062 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:19:53.165997 10075 raft_consensus.cc:729] T ba256336804d425ab1b4f5e8aa1f2b45 P 629a4463a87648bfabc61517716e9062 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 629a4463a87648bfabc61517716e9062, State: Initialized, Role: FOLLOWER
I20260809 02:19:53.166281 10075 consensus_queue.cc:260] T ba256336804d425ab1b4f5e8aa1f2b45 P 629a4463a87648bfabc61517716e9062 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b379803659d2405f8826b28a044dde77" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 42105 } } peers { permanent_uuid: "629a4463a87648bfabc61517716e9062" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 33115 } } peers { permanent_uuid: "4a28c730071b43f2a39b62fcf3915d51" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 38761 } }
I20260809 02:19:53.166815 10045 heartbeater.cc:503] Master 127.15.75.190:41203 was elected leader, sending a full tablet report...
I20260809 02:19:53.167006 10075 ts_tablet_manager.cc:1428] T ba256336804d425ab1b4f5e8aa1f2b45 P 629a4463a87648bfabc61517716e9062: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260809 02:19:53.249455 10084 raft_consensus.cc:482] T ba256336804d425ab1b4f5e8aa1f2b45 P 629a4463a87648bfabc61517716e9062 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:19:53.249652 10084 raft_consensus.cc:504] T ba256336804d425ab1b4f5e8aa1f2b45 P 629a4463a87648bfabc61517716e9062 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b379803659d2405f8826b28a044dde77" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 42105 } } peers { permanent_uuid: "629a4463a87648bfabc61517716e9062" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 33115 } } peers { permanent_uuid: "4a28c730071b43f2a39b62fcf3915d51" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 38761 } }
I20260809 02:19:53.250813 10084 leader_election.cc:288] T ba256336804d425ab1b4f5e8aa1f2b45 P 629a4463a87648bfabc61517716e9062 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b379803659d2405f8826b28a044dde77 (127.15.75.130:42105), 4a28c730071b43f2a39b62fcf3915d51 (127.15.75.129:38761)
I20260809 02:19:53.255336 10064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:53.251141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.131:45087 (local address 127.15.75.130:42105)
0809 02:19:53.251278 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:53.251283 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:53.251300 (+ 17us) server_negotiation.cc:407] Connection header received
0809 02:19:53.251432 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:53.251435 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:53.251494 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:53.251589 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:19:53.251959 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:53.252660 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:53.253933 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:53.254569 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:53.254713 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:53.254768 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:53.255136 (+ 368us) server_negotiation.cc:299] Negotiation successful
0809 02:19:53.255210 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 02:19:53.255336 10072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:53.251237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.131:51139 (local address 127.15.75.129:38761)
0809 02:19:53.251366 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:53.251371 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:53.251623 (+ 252us) server_negotiation.cc:407] Connection header received
0809 02:19:53.251738 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:53.251741 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:53.251795 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:53.251895 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:19:53.252278 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:53.252824 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:53.254031 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:53.254625 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:53.254732 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:53.254773 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:53.255136 (+ 363us) server_negotiation.cc:299] Negotiation successful
0809 02:19:53.255210 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 02:19:53.255339 10047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:53.251033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.130:42105 (local address 127.15.75.131:45087)
0809 02:19:53.251208 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:53.251224 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 02:19:53.251316 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:19:53.251625 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:19:53.251628 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:19:53.251651 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:19:53.251806 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:53.251815 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:53.252817 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:53.252820 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:53.253804 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:53.253814 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:53.254766 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:53.254770 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:53.254967 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:53.255020 (+ 53us) client_negotiation.cc:769] Sending connection context
0809 02:19:53.255095 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 02:19:53.255188 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"mutex_wait_us":67}
I20260809 02:19:53.255349 10085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:53.251073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.129:38761 (local address 127.15.75.131:51139)
0809 02:19:53.251527 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:53.251542 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:19:53.251637 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:19:53.251957 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:19:53.251961 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:19:53.251981 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:19:53.252106 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:53.252112 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:53.252958 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:53.252960 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:53.253896 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:53.253908 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:53.254766 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:53.254770 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:53.254974 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:53.255011 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 02:19:53.255095 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 02:19:53.255188 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":387,"mutex_wait_us":80,"thread_start_us":150,"threads_started":1}
I20260809 02:19:53.256090 9762 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba256336804d425ab1b4f5e8aa1f2b45" candidate_uuid: "629a4463a87648bfabc61517716e9062" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b379803659d2405f8826b28a044dde77" is_pre_election: true
I20260809 02:19:53.256240 9549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba256336804d425ab1b4f5e8aa1f2b45" candidate_uuid: "629a4463a87648bfabc61517716e9062" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a28c730071b43f2a39b62fcf3915d51" is_pre_election: true
I20260809 02:19:53.256420 9762 raft_consensus.cc:2454] T ba256336804d425ab1b4f5e8aa1f2b45 P b379803659d2405f8826b28a044dde77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 629a4463a87648bfabc61517716e9062 in term 0.
I20260809 02:19:53.256582 9549 raft_consensus.cc:2454] T ba256336804d425ab1b4f5e8aa1f2b45 P 4a28c730071b43f2a39b62fcf3915d51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 629a4463a87648bfabc61517716e9062 in term 0.
I20260809 02:19:53.257009 9866 leader_election.cc:302] T ba256336804d425ab1b4f5e8aa1f2b45 P 629a4463a87648bfabc61517716e9062 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 4a28c730071b43f2a39b62fcf3915d51, 629a4463a87648bfabc61517716e9062, b379803659d2405f8826b28a044dde77; no voters:
I20260809 02:19:53.257311 10084 raft_consensus.cc:2781] T ba256336804d425ab1b4f5e8aa1f2b45 P 629a4463a87648bfabc61517716e9062 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:19:53.257397 10084 raft_consensus.cc:482] T ba256336804d425ab1b4f5e8aa1f2b45 P 629a4463a87648bfabc61517716e9062 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:19:53.257431 10084 raft_consensus.cc:3037] T ba256336804d425ab1b4f5e8aa1f2b45 P 629a4463a87648bfabc61517716e9062 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:19:53.258450 10084 raft_consensus.cc:504] T ba256336804d425ab1b4f5e8aa1f2b45 P 629a4463a87648bfabc61517716e9062 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b379803659d2405f8826b28a044dde77" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 42105 } } peers { permanent_uuid: "629a4463a87648bfabc61517716e9062" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 33115 } } peers { permanent_uuid: "4a28c730071b43f2a39b62fcf3915d51" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 38761 } }
I20260809 02:19:53.258745 10084 leader_election.cc:288] T ba256336804d425ab1b4f5e8aa1f2b45 P 629a4463a87648bfabc61517716e9062 [CANDIDATE]: Term 1 election: Requested vote from peers b379803659d2405f8826b28a044dde77 (127.15.75.130:42105), 4a28c730071b43f2a39b62fcf3915d51 (127.15.75.129:38761)
I20260809 02:19:53.259183 9549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba256336804d425ab1b4f5e8aa1f2b45" candidate_uuid: "629a4463a87648bfabc61517716e9062" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a28c730071b43f2a39b62fcf3915d51"
I20260809 02:19:53.259291 9549 raft_consensus.cc:3037] T ba256336804d425ab1b4f5e8aa1f2b45 P 4a28c730071b43f2a39b62fcf3915d51 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:19:53.260496 9549 raft_consensus.cc:2454] T ba256336804d425ab1b4f5e8aa1f2b45 P 4a28c730071b43f2a39b62fcf3915d51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 629a4463a87648bfabc61517716e9062 in term 1.
I20260809 02:19:53.260823 9762 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba256336804d425ab1b4f5e8aa1f2b45" candidate_uuid: "629a4463a87648bfabc61517716e9062" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b379803659d2405f8826b28a044dde77"
I20260809 02:19:53.260923 9867 leader_election.cc:302] T ba256336804d425ab1b4f5e8aa1f2b45 P 629a4463a87648bfabc61517716e9062 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a28c730071b43f2a39b62fcf3915d51, 629a4463a87648bfabc61517716e9062; no voters:
I20260809 02:19:53.260957 9762 raft_consensus.cc:3037] T ba256336804d425ab1b4f5e8aa1f2b45 P b379803659d2405f8826b28a044dde77 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:19:53.261098 10084 raft_consensus.cc:2781] T ba256336804d425ab1b4f5e8aa1f2b45 P 629a4463a87648bfabc61517716e9062 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:19:53.261387 10084 raft_consensus.cc:686] T ba256336804d425ab1b4f5e8aa1f2b45 P 629a4463a87648bfabc61517716e9062 [term 1 LEADER]: Becoming Leader. State: Replica: 629a4463a87648bfabc61517716e9062, State: Running, Role: LEADER
I20260809 02:19:53.261746 10084 consensus_queue.cc:237] T ba256336804d425ab1b4f5e8aa1f2b45 P 629a4463a87648bfabc61517716e9062 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b379803659d2405f8826b28a044dde77" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 42105 } } peers { permanent_uuid: "629a4463a87648bfabc61517716e9062" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 33115 } } peers { permanent_uuid: "4a28c730071b43f2a39b62fcf3915d51" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 38761 } }
I20260809 02:19:53.262214 9762 raft_consensus.cc:2454] T ba256336804d425ab1b4f5e8aa1f2b45 P b379803659d2405f8826b28a044dde77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 629a4463a87648bfabc61517716e9062 in term 1.
I20260809 02:19:53.264988 9330 catalog_manager.cc:5495] T ba256336804d425ab1b4f5e8aa1f2b45 P 629a4463a87648bfabc61517716e9062 reported cstate change: term changed from 0 to 1, leader changed from <none> to 629a4463a87648bfabc61517716e9062 (127.15.75.131). New cstate: current_term: 1 leader_uuid: "629a4463a87648bfabc61517716e9062" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b379803659d2405f8826b28a044dde77" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 42105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "629a4463a87648bfabc61517716e9062" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 33115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a28c730071b43f2a39b62fcf3915d51" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 38761 } health_report { overall_health: UNKNOWN } } }
I20260809 02:19:53.292160 10069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:53.288425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41062 (local address 127.15.75.131:33115)
0809 02:19:53.288567 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:53.288571 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:53.288586 (+ 15us) server_negotiation.cc:407] Connection header received
0809 02:19:53.288770 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:53.288773 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:53.288843 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:53.288961 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:19:53.289228 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:53.289749 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:53.290643 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:53.290880 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:53.290995 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:53.291320 (+ 325us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:19:53.291732 (+ 412us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:19:53.291869 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:53.291929 (+ 60us) server_negotiation.cc:299] Negotiation successful
0809 02:19:53.291987 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 02:19:53.298133 9549 raft_consensus.cc:1261] T ba256336804d425ab1b4f5e8aa1f2b45 P 4a28c730071b43f2a39b62fcf3915d51 [term 1 FOLLOWER]: Refusing update from remote peer 629a4463a87648bfabc61517716e9062: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:19:53.298133 9762 raft_consensus.cc:1261] T ba256336804d425ab1b4f5e8aa1f2b45 P b379803659d2405f8826b28a044dde77 [term 1 FOLLOWER]: Refusing update from remote peer 629a4463a87648bfabc61517716e9062: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:19:53.298892 10086 consensus_queue.cc:1035] T ba256336804d425ab1b4f5e8aa1f2b45 P 629a4463a87648bfabc61517716e9062 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a28c730071b43f2a39b62fcf3915d51" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 38761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:19:53.299039 10084 consensus_queue.cc:1035] T ba256336804d425ab1b4f5e8aa1f2b45 P 629a4463a87648bfabc61517716e9062 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b379803659d2405f8826b28a044dde77" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 42105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:19:53.307091 10096 mvcc.cc:205] Tried to move back new op lower bound from 7316447204541210624 to 7316447204403118080. Current Snapshot: MvccSnapshot[applied={T|T < 7316447204541210624}]
I20260809 02:19:53.314941 9330 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59256:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:19:53.315124 9330 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59256:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:19:53.317687 9330 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7e9165c554dc4389984130d40adcfc31: Sending DeleteTablet for 3 replicas of tablet ba256336804d425ab1b4f5e8aa1f2b45
I20260809 02:19:53.318434 9733 tablet_service.cc:1514] Processing DeleteTablet for tablet ba256336804d425ab1b4f5e8aa1f2b45 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:19:53 UTC) from {username='kudu', principal='kudu/127.15.75.190@KRBTEST.COM'} at 127.0.0.1:34244
I20260809 02:19:53.318629 9942 tablet_service.cc:1514] Processing DeleteTablet for tablet ba256336804d425ab1b4f5e8aa1f2b45 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:19:53 UTC) from {username='kudu', principal='kudu/127.15.75.190@KRBTEST.COM'} at 127.0.0.1:41056
I20260809 02:19:53.319168 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 9416
I20260809 02:19:53.319258 9522 tablet_service.cc:1514] Processing DeleteTablet for tablet ba256336804d425ab1b4f5e8aa1f2b45 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:19:53 UTC) from {username='kudu', principal='kudu/127.15.75.190@KRBTEST.COM'} at 127.0.0.1:34098
I20260809 02:19:53.326689 10109 tablet_replica.cc:330] T ba256336804d425ab1b4f5e8aa1f2b45 P 629a4463a87648bfabc61517716e9062: stopping tablet replica
I20260809 02:19:53.327106 10109 raft_consensus.cc:2229] T ba256336804d425ab1b4f5e8aa1f2b45 P 629a4463a87648bfabc61517716e9062 [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:19:53.327473 10107 tablet_replica.cc:330] T ba256336804d425ab1b4f5e8aa1f2b45 P b379803659d2405f8826b28a044dde77: stopping tablet replica
I20260809 02:19:53.327899 10107 raft_consensus.cc:2229] T ba256336804d425ab1b4f5e8aa1f2b45 P b379803659d2405f8826b28a044dde77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:19:53.328248 10107 raft_consensus.cc:2258] T ba256336804d425ab1b4f5e8aa1f2b45 P b379803659d2405f8826b28a044dde77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:19:53.328898 10109 raft_consensus.cc:2258] T ba256336804d425ab1b4f5e8aa1f2b45 P 629a4463a87648bfabc61517716e9062 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:19:53.330364 10109 ts_tablet_manager.cc:1905] T ba256336804d425ab1b4f5e8aa1f2b45 P 629a4463a87648bfabc61517716e9062: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:19:53.334002 10107 ts_tablet_manager.cc:1905] T ba256336804d425ab1b4f5e8aa1f2b45 P b379803659d2405f8826b28a044dde77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:19:53.334023 10109 ts_tablet_manager.cc:1918] T ba256336804d425ab1b4f5e8aa1f2b45 P 629a4463a87648bfabc61517716e9062: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:19:53.334117 10109 log.cc:1196] T ba256336804d425ab1b4f5e8aa1f2b45 P 629a4463a87648bfabc61517716e9062: Deleting WAL directory at /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/minicluster-data/ts-2/wal/wals/ba256336804d425ab1b4f5e8aa1f2b45
I20260809 02:19:53.334591 10109 ts_tablet_manager.cc:1939] T ba256336804d425ab1b4f5e8aa1f2b45 P 629a4463a87648bfabc61517716e9062: Deleting consensus metadata
W20260809 02:19:53.335649 9304 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.75.129:38761 (error 108)
I20260809 02:19:53.336199 9304 catalog_manager.cc:4841] TS 629a4463a87648bfabc61517716e9062 (127.15.75.131:33115): tablet ba256336804d425ab1b4f5e8aa1f2b45 (table test-table [id=fe12f57cc29f45e3af04e5c74e625c0a]) successfully deleted
I20260809 02:19:53.336743 10107 ts_tablet_manager.cc:1918] T ba256336804d425ab1b4f5e8aa1f2b45 P b379803659d2405f8826b28a044dde77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260809 02:19:53.336897 10067 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:19:53.336518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.129:38761 (local address 127.0.0.1:34110)
0809 02:19:53.336656 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:53.336675 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 02:19:53.336777 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.129:38761: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":54}
I20260809 02:19:53.336911 10107 log.cc:1196] T ba256336804d425ab1b4f5e8aa1f2b45 P b379803659d2405f8826b28a044dde77: Deleting WAL directory at /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241741788589-15662-0/minicluster-data/ts-1/wal/wals/ba256336804d425ab1b4f5e8aa1f2b45
I20260809 02:19:53.337144 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 9635
I20260809 02:19:53.337229 10107 ts_tablet_manager.cc:1939] T ba256336804d425ab1b4f5e8aa1f2b45 P b379803659d2405f8826b28a044dde77: Deleting consensus metadata
W20260809 02:19:53.337199 9304 catalog_manager.cc:4584] TS 4a28c730071b43f2a39b62fcf3915d51 (127.15.75.129:38761): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ba256336804d425ab1b4f5e8aa1f2b45: Network error: Client connection negotiation failed: client connection to 127.15.75.129:38761: BlockingWrite error: write error: Connection reset by peer (error 104)
I20260809 02:19:53.346712 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 9839
I20260809 02:19:53.346927 10067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:53.346605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.130:42105 (local address 127.0.0.1:34260)
0809 02:19:53.346728 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:53.346826 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.130:42105: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260809 02:19:53.356202 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 9275
2026-08-09T02:19:53Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2536 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-09T02:19:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:19:53Z Disabled control of system clock
I20260809 02:19:53.392866 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241741788589-15662-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241741788589-15662-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241741788589-15662-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:39607
--webserver_interface=127.15.75.190
--webserver_port=0
--builtin_ntp_servers=127.15.75.148:38321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.75.190:39607
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 02:19:53.504070 10114 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:19:53.504299 10114 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:19:53.504333 10114 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:19:53.508919 10114 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:19:53.508972 10114 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:19:53.508987 10114 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:19:53.509004 10114 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:19:53.509019 10114 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:19:53.513556 10114 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:38321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241741788589-15662-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241741788589-15662-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.75.190:39607
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:39607
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.75.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10114
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241741788589-15662-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:19:53.514460 10114 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:19:53.515709 10114 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:19:53.524168 10121 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:19:53.524178 10122 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:19:53.524209 10124 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:19:53.524495 10114 server_base.cc:1034] running on GCE node
I20260809 02:19:53.525125 10114 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:19:53.526193 10114 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:19:53.527359 10114 hybrid_clock.cc:648] HybridClock initialized: now 1786241993527339 us; error 29 us; skew 500 ppm
I20260809 02:19:53.529302 10114 webserver.cc:460] Webserver started at http://127.15.75.190:33231/ using document root <none> and password file <none>
I20260809 02:19:53.529891 10114 fs_manager.cc:359] Metadata directory not provided
I20260809 02:19:53.529956 10114 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:19:53.530205 10114 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:19:53.532254 10114 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241741788589-15662-0/minicluster-data/master-0/data/instance:
uuid: "1a8c954996b24d0e856a6c63fd7a02f6"
format_stamp: "Formatted at 2026-08-09 02:19:53 on dist-test-slave-s11t"
server_key: "6b70654e4d02111cae6c4c53fb054b94"
server_key_iv: "072f472ba4edfcfe18e60f136abeaf9e"
server_key_version: "encryptionkey@0"
I20260809 02:19:53.532860 10114 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241741788589-15662-0/minicluster-data/master-0/wal/instance:
uuid: "1a8c954996b24d0e856a6c63fd7a02f6"
format_stamp: "Formatted at 2026-08-09 02:19:53 on dist-test-slave-s11t"
server_key: "6b70654e4d02111cae6c4c53fb054b94"
server_key_iv: "072f472ba4edfcfe18e60f136abeaf9e"
server_key_version: "encryptionkey@0"
I20260809 02:19:53.536450 10114 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260809 02:19:53.538928 10132 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:19:53.540107 10114 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 02:19:53.540248 10114 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241741788589-15662-0/minicluster-data/master-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241741788589-15662-0/minicluster-data/master-0/wal
uuid: "1a8c954996b24d0e856a6c63fd7a02f6"
format_stamp: "Formatted at 2026-08-09 02:19:53 on dist-test-slave-s11t"
server_key: "6b70654e4d02111cae6c4c53fb054b94"
server_key_iv: "072f472ba4edfcfe18e60f136abeaf9e"
server_key_version: "encryptionkey@0"
I20260809 02:19:53.540359 10114 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241741788589-15662-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241741788589-15662-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241741788589-15662-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:19:53.558709 10114 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:19:53.559335 10114 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:19:53.559551 10114 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:19:53.566514 10208 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.190:39607 every 8 connection(s)
I20260809 02:19:53.566510 10114 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.190:39607
I20260809 02:19:53.567447 10114 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
I20260809 02:19:53.568593 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 10114
I20260809 02:19:53.568754 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241741788589-15662-0/minicluster-data/master-0/wal/instance
I20260809 02:19:53.569064 15662 external_mini_cluster.cc:1428] Setting key 415a4f6467283b3684466679d12f61be
I20260809 02:19:53.571079 10210 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:19:53.576440 10210 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a8c954996b24d0e856a6c63fd7a02f6: Bootstrap starting.
I20260809 02:19:53.578145 10213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:53.570403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52240 (local address 127.15.75.190:39607)
0809 02:19:53.571048 (+ 645us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:53.571062 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:53.571096 (+ 34us) server_negotiation.cc:407] Connection header received
0809 02:19:53.571777 (+ 681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:53.571793 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:53.572169 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:53.572504 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:53.572967 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:53.573976 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:53.574882 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:53.575349 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:53.575534 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:53.576421 (+ 887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:19:53.576426 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:19:53.576438 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:19:53.576924 (+ 486us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:19:53.576979 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:19:53.577004 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:19:53.577117 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:53.577237 (+ 120us) server_negotiation.cc:299] Negotiation successful
0809 02:19:53.577497 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":164,"threads_started":1}
I20260809 02:19:53.578676 10210 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1a8c954996b24d0e856a6c63fd7a02f6: Neither blocks nor log segments found. Creating new log.
I20260809 02:19:53.579531 10210 log.cc:824] T 00000000000000000000000000000000 P 1a8c954996b24d0e856a6c63fd7a02f6: Log is configured to *not* fsync() on all Append() calls
I20260809 02:19:53.581475 10210 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a8c954996b24d0e856a6c63fd7a02f6: No bootstrap required, opened a new log
I20260809 02:19:53.585052 10210 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a8c954996b24d0e856a6c63fd7a02f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a8c954996b24d0e856a6c63fd7a02f6" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 39607 } }
I20260809 02:19:53.585249 10210 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1a8c954996b24d0e856a6c63fd7a02f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:19:53.585291 10210 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a8c954996b24d0e856a6c63fd7a02f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a8c954996b24d0e856a6c63fd7a02f6, State: Initialized, Role: FOLLOWER
I20260809 02:19:53.585620 10210 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a8c954996b24d0e856a6c63fd7a02f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a8c954996b24d0e856a6c63fd7a02f6" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 39607 } }
I20260809 02:19:53.585723 10210 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1a8c954996b24d0e856a6c63fd7a02f6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:19:53.585789 10210 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a8c954996b24d0e856a6c63fd7a02f6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:19:53.585861 10210 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a8c954996b24d0e856a6c63fd7a02f6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:19:53.586602 10210 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a8c954996b24d0e856a6c63fd7a02f6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a8c954996b24d0e856a6c63fd7a02f6" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 39607 } }
I20260809 02:19:53.586877 10210 leader_election.cc:302] T 00000000000000000000000000000000 P 1a8c954996b24d0e856a6c63fd7a02f6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1a8c954996b24d0e856a6c63fd7a02f6; no voters:
I20260809 02:19:53.587141 10210 leader_election.cc:288] T 00000000000000000000000000000000 P 1a8c954996b24d0e856a6c63fd7a02f6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:19:53.587234 10215 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a8c954996b24d0e856a6c63fd7a02f6 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:19:53.587486 10215 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1a8c954996b24d0e856a6c63fd7a02f6 [term 1 LEADER]: Becoming Leader. State: Replica: 1a8c954996b24d0e856a6c63fd7a02f6, State: Running, Role: LEADER
I20260809 02:19:53.587845 10215 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a8c954996b24d0e856a6c63fd7a02f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a8c954996b24d0e856a6c63fd7a02f6" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 39607 } }
I20260809 02:19:53.588312 10210 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a8c954996b24d0e856a6c63fd7a02f6 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:19:53.589538 10217 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a8c954996b24d0e856a6c63fd7a02f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a8c954996b24d0e856a6c63fd7a02f6. Latest consensus state: current_term: 1 leader_uuid: "1a8c954996b24d0e856a6c63fd7a02f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a8c954996b24d0e856a6c63fd7a02f6" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 39607 } } }
I20260809 02:19:53.589664 10217 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a8c954996b24d0e856a6c63fd7a02f6 [sys.catalog]: This master's current role is: LEADER
I20260809 02:19:53.589962 10216 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a8c954996b24d0e856a6c63fd7a02f6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1a8c954996b24d0e856a6c63fd7a02f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a8c954996b24d0e856a6c63fd7a02f6" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 39607 } } }
I20260809 02:19:53.590058 10216 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a8c954996b24d0e856a6c63fd7a02f6 [sys.catalog]: This master's current role is: LEADER
I20260809 02:19:53.590087 10230 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:19:53.593171 10230 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:19:53.598855 10230 catalog_manager.cc:1347] Generated new cluster ID: fb7c879721dd4d23b280087e37c2f314
I20260809 02:19:53.598935 10230 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:19:53.627918 10230 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:19:53.628925 10230 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:19:53.634042 10230 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1a8c954996b24d0e856a6c63fd7a02f6: Generated new TSK 0
I20260809 02:19:53.634745 10230 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:19:53.651171 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241741788589-15662-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241741788589-15662-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241741788589-15662-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.129:0
--local_ip_for_outbound_sockets=127.15.75.129
--webserver_interface=127.15.75.129
--webserver_port=0
--tserver_master_addrs=127.15.75.190:39607
--builtin_ntp_servers=127.15.75.148:38321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 02:19:53.776407 10248 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:19:53.776654 10248 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:19:53.776773 10248 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:19:53.780654 10248 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:19:53.780769 10248 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:19:53.780825 10248 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.129
I20260809 02:19:53.785457 10248 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:38321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241741788589-15662-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241741788589-15662-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.75.129
--webserver_port=0
--tserver_master_addrs=127.15.75.190:39607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10248
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.129
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241741788589-15662-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:19:53.786417 10248 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:19:53.787724 10248 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:19:53.797216 10255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:19:53.797235 10254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:19:53.797230 10258 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:19:53.797689 10248 server_base.cc:1034] running on GCE node
I20260809 02:19:53.798079 10248 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:19:53.798683 10248 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:19:53.799891 10248 hybrid_clock.cc:648] HybridClock initialized: now 1786241993799854 us; error 45 us; skew 500 ppm
I20260809 02:19:53.801704 10248 webserver.cc:460] Webserver started at http://127.15.75.129:43269/ using document root <none> and password file <none>
I20260809 02:19:53.802278 10248 fs_manager.cc:359] Metadata directory not provided
I20260809 02:19:53.802372 10248 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:19:53.802603 10248 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:19:53.804574 10248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241741788589-15662-0/minicluster-data/ts-0/data/instance:
uuid: "95e8d4837a734996b9df4059e4a07e72"
format_stamp: "Formatted at 2026-08-09 02:19:53 on dist-test-slave-s11t"
server_key: "9d2dc5b2e0a33ef5fb5cc59485e80db9"
server_key_iv: "bf9109fdadc319aff7139204381dc1fd"
server_key_version: "encryptionkey@0"
I20260809 02:19:53.805205 10248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241741788589-15662-0/minicluster-data/ts-0/wal/instance:
uuid: "95e8d4837a734996b9df4059e4a07e72"
format_stamp: "Formatted at 2026-08-09 02:19:53 on dist-test-slave-s11t"
server_key: "9d2dc5b2e0a33ef5fb5cc59485e80db9"
server_key_iv: "bf9109fdadc319aff7139204381dc1fd"
server_key_version: "encryptionkey@0"
I20260809 02:19:53.808614 10248 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 02:19:53.811080 10269 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:19:53.812351 10248 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 02:19:53.812444 10248 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241741788589-15662-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241741788589-15662-0/minicluster-data/ts-0/wal
uuid: "95e8d4837a734996b9df4059e4a07e72"
format_stamp: "Formatted at 2026-08-09 02:19:53 on dist-test-slave-s11t"
server_key: "9d2dc5b2e0a33ef5fb5cc59485e80db9"
server_key_iv: "bf9109fdadc319aff7139204381dc1fd"
server_key_version: "encryptionkey@0"
I20260809 02:19:53.812577 10248 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241741788589-15662-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241741788589-15662-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241741788589-15662-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:19:53.833662 10248 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:19:53.834354 10248 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:19:53.834559 10248 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:19:53.835158 10248 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:19:53.836298 10248 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:19:53.836354 10248 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:19:53.836428 10248 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:19:53.836467 10248 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:19:53.845989 10248 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.129:44415
I20260809 02:19:53.846021 10441 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.129:44415 every 8 connection(s)
I20260809 02:19:53.846887 10248 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
I20260809 02:19:53.849231 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 10248
I20260809 02:19:53.849375 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241741788589-15662-0/minicluster-data/ts-0/wal/instance
I20260809 02:19:53.849716 15662 external_mini_cluster.cc:1428] Setting key b707ef98ca8914dfd176efbeafc22793
I20260809 02:19:53.851902 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241741788589-15662-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241741788589-15662-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241741788589-15662-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.130:0
--local_ip_for_outbound_sockets=127.15.75.130
--webserver_interface=127.15.75.130
--webserver_port=0
--tserver_master_addrs=127.15.75.190:39607
--builtin_ntp_servers=127.15.75.148:38321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 02:19:53.857970 10213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:53.848701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.129:53475 (local address 127.15.75.190:39607)
0809 02:19:53.848889 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:53.848894 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:53.849982 (+ 1088us) server_negotiation.cc:407] Connection header received
0809 02:19:53.851582 (+ 1600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:53.851587 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:53.851651 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:53.851756 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:53.852720 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:53.853706 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:53.855652 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:53.856054 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:53.856182 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:53.857340 (+ 1158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:19:53.857345 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:19:53.857348 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:19:53.857364 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:19:53.857391 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:19:53.857404 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:19:53.857492 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:53.857652 (+ 160us) server_negotiation.cc:299] Negotiation successful
0809 02:19:53.857761 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 02:19:53.858647 10444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:53.849117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:39607 (local address 127.15.75.129:53475)
0809 02:19:53.849813 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:53.849865 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 02:19:53.851374 (+ 1509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:19:53.851931 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:19:53.851938 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:19:53.852047 (+ 109us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:19:53.852547 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:53.852560 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:53.853890 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:53.853897 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:53.855418 (+ 1521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:53.855437 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:53.856224 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:53.856227 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:53.856371 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:53.857067 (+ 696us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:19:53.857069 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:19:53.857107 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:19:53.857169 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:19:53.857502 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:19:53.857511 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:19:53.857516 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 02:19:53.857618 (+ 102us) client_negotiation.cc:240] Negotiation successful
0809 02:19:53.857840 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":135,"threads_started":1}
I20260809 02:19:53.859943 10442 heartbeater.cc:347] Connected to a master server at 127.15.75.190:39607
I20260809 02:19:53.860201 10442 heartbeater.cc:464] Registering TS with master...
I20260809 02:19:53.860891 10442 heartbeater.cc:511] Master 127.15.75.190:39607 requested a full tablet report, sending...
I20260809 02:19:53.862473 10156 ts_manager.cc:194] Registered new tserver with Master: 95e8d4837a734996b9df4059e4a07e72 (127.15.75.129:44415)
I20260809 02:19:53.863619 10156 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.75.129:53475
W20260809 02:19:53.966329 10445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:19:53.966564 10445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:19:53.966600 10445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:19:53.970722 10445 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:19:53.970778 10445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:19:53.970851 10445 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.130
I20260809 02:19:53.975639 10445 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:38321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241741788589-15662-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241741788589-15662-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.75.130
--webserver_port=0
--tserver_master_addrs=127.15.75.190:39607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10445
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.130
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241741788589-15662-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:19:53.976622 10445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:19:53.977965 10445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:19:53.987891 10456 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:19:53.987982 10445 server_base.cc:1034] running on GCE node
W20260809 02:19:53.987942 10454 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:19:53.987891 10460 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:19:53.988548 10445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:19:53.989302 10445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:19:53.990514 10445 hybrid_clock.cc:648] HybridClock initialized: now 1786241993990493 us; error 26 us; skew 500 ppm
I20260809 02:19:53.992319 10445 webserver.cc:460] Webserver started at http://127.15.75.130:40715/ using document root <none> and password file <none>
I20260809 02:19:53.992921 10445 fs_manager.cc:359] Metadata directory not provided
I20260809 02:19:53.992985 10445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:19:53.993151 10445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:19:53.995289 10445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241741788589-15662-0/minicluster-data/ts-1/data/instance:
uuid: "e8e12df1d8a64b20a72071cfe0b9fc83"
format_stamp: "Formatted at 2026-08-09 02:19:53 on dist-test-slave-s11t"
server_key: "61c8845e24a7bf7a4707f1cf5364be14"
server_key_iv: "69186a3e515d9f618b51410a87493bc3"
server_key_version: "encryptionkey@0"
I20260809 02:19:53.995790 10445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241741788589-15662-0/minicluster-data/ts-1/wal/instance:
uuid: "e8e12df1d8a64b20a72071cfe0b9fc83"
format_stamp: "Formatted at 2026-08-09 02:19:53 on dist-test-slave-s11t"
server_key: "61c8845e24a7bf7a4707f1cf5364be14"
server_key_iv: "69186a3e515d9f618b51410a87493bc3"
server_key_version: "encryptionkey@0"
I20260809 02:19:53.999341 10445 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260809 02:19:54.001832 10466 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:19:54.003161 10445 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 02:19:54.003328 10445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241741788589-15662-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241741788589-15662-0/minicluster-data/ts-1/wal
uuid: "e8e12df1d8a64b20a72071cfe0b9fc83"
format_stamp: "Formatted at 2026-08-09 02:19:53 on dist-test-slave-s11t"
server_key: "61c8845e24a7bf7a4707f1cf5364be14"
server_key_iv: "69186a3e515d9f618b51410a87493bc3"
server_key_version: "encryptionkey@0"
I20260809 02:19:54.003468 10445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241741788589-15662-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241741788589-15662-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241741788589-15662-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:19:54.025848 10445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:19:54.026517 10445 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:19:54.026717 10445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:19:54.027309 10445 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:19:54.028256 10445 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:19:54.028331 10445 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:19:54.028405 10445 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:19:54.028448 10445 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:19:54.036883 10445 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.130:38099
I20260809 02:19:54.036952 10636 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.130:38099 every 8 connection(s)
I20260809 02:19:54.037796 10445 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
I20260809 02:19:54.040088 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 10445
I20260809 02:19:54.040220 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241741788589-15662-0/minicluster-data/ts-1/wal/instance
I20260809 02:19:54.040584 15662 external_mini_cluster.cc:1428] Setting key 4be2ae740e8d95506d2ddbe5794e943e
I20260809 02:19:54.042706 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241741788589-15662-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241741788589-15662-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241741788589-15662-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.131:0
--local_ip_for_outbound_sockets=127.15.75.131
--webserver_interface=127.15.75.131
--webserver_port=0
--tserver_master_addrs=127.15.75.190:39607
--builtin_ntp_servers=127.15.75.148:38321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 02:19:54.048378 10213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:54.039636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.130:46301 (local address 127.15.75.190:39607)
0809 02:19:54.039792 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:54.039797 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:54.041217 (+ 1420us) server_negotiation.cc:407] Connection header received
0809 02:19:54.042953 (+ 1736us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:54.042956 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:54.043005 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:54.043051 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:54.044224 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:54.044927 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:54.046212 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:54.046468 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:54.046585 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:54.047812 (+ 1227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:19:54.047816 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:19:54.047819 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:19:54.047836 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:19:54.047863 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:19:54.047873 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:19:54.047964 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:54.048141 (+ 177us) server_negotiation.cc:299] Negotiation successful
0809 02:19:54.048213 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 02:19:54.049459 10639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:54.040153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:39607 (local address 127.15.75.130:46301)
0809 02:19:54.041015 (+ 862us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:54.041073 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 02:19:54.042636 (+ 1563us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:19:54.043390 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:19:54.043396 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:19:54.043482 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:19:54.044062 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:54.044077 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:54.045073 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:54.045077 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:54.046059 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:54.046069 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:54.046637 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:54.046642 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:54.046835 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:54.047527 (+ 692us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:19:54.047530 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:19:54.047571 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:19:54.047639 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:19:54.047966 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:19:54.047979 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:19:54.047988 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 02:19:54.048140 (+ 152us) client_negotiation.cc:240] Negotiation successful
0809 02:19:54.048438 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":155,"threads_started":1}
I20260809 02:19:54.050689 10637 heartbeater.cc:347] Connected to a master server at 127.15.75.190:39607
I20260809 02:19:54.050962 10637 heartbeater.cc:464] Registering TS with master...
I20260809 02:19:54.051647 10637 heartbeater.cc:511] Master 127.15.75.190:39607 requested a full tablet report, sending...
I20260809 02:19:54.052997 10156 ts_manager.cc:194] Registered new tserver with Master: e8e12df1d8a64b20a72071cfe0b9fc83 (127.15.75.130:38099)
I20260809 02:19:54.053638 10156 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.75.130:46301
W20260809 02:19:54.156562 10640 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:19:54.156855 10640 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:19:54.156925 10640 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:19:54.160892 10640 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:19:54.160936 10640 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:19:54.161011 10640 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.131
I20260809 02:19:54.165647 10640 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:38321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241741788589-15662-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241741788589-15662-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.75.131
--webserver_port=0
--tserver_master_addrs=127.15.75.190:39607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10640
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.131
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241741788589-15662-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:19:54.166567 10640 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:19:54.167838 10640 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 02:19:54.177317 10640 server_base.cc:1034] running on GCE node
W20260809 02:19:54.177274 10647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:19:54.177284 10648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:19:54.177284 10650 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:19:54.177959 10640 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:19:54.178561 10640 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:19:54.179756 10640 hybrid_clock.cc:648] HybridClock initialized: now 1786241994179732 us; error 45 us; skew 500 ppm
I20260809 02:19:54.181614 10640 webserver.cc:460] Webserver started at http://127.15.75.131:42955/ using document root <none> and password file <none>
I20260809 02:19:54.182174 10640 fs_manager.cc:359] Metadata directory not provided
I20260809 02:19:54.182274 10640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:19:54.182505 10640 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:19:54.184492 10640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241741788589-15662-0/minicluster-data/ts-2/data/instance:
uuid: "fae77d282a5f495fbd8fc9e9ab5f3278"
format_stamp: "Formatted at 2026-08-09 02:19:54 on dist-test-slave-s11t"
server_key: "0ca9d232531f8566f300f021056ed095"
server_key_iv: "d23d19bf0ec59fff40e33b389d183901"
server_key_version: "encryptionkey@0"
I20260809 02:19:54.185108 10640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241741788589-15662-0/minicluster-data/ts-2/wal/instance:
uuid: "fae77d282a5f495fbd8fc9e9ab5f3278"
format_stamp: "Formatted at 2026-08-09 02:19:54 on dist-test-slave-s11t"
server_key: "0ca9d232531f8566f300f021056ed095"
server_key_iv: "d23d19bf0ec59fff40e33b389d183901"
server_key_version: "encryptionkey@0"
I20260809 02:19:54.188619 10640 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260809 02:19:54.191061 10659 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:19:54.192167 10640 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 02:19:54.192313 10640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241741788589-15662-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241741788589-15662-0/minicluster-data/ts-2/wal
uuid: "fae77d282a5f495fbd8fc9e9ab5f3278"
format_stamp: "Formatted at 2026-08-09 02:19:54 on dist-test-slave-s11t"
server_key: "0ca9d232531f8566f300f021056ed095"
server_key_iv: "d23d19bf0ec59fff40e33b389d183901"
server_key_version: "encryptionkey@0"
I20260809 02:19:54.192431 10640 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241741788589-15662-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241741788589-15662-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241741788589-15662-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:19:54.218827 10640 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:19:54.219559 10640 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:19:54.219759 10640 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:19:54.220343 10640 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:19:54.221292 10640 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:19:54.221372 10640 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:19:54.221449 10640 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:19:54.221495 10640 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:19:54.230628 10640 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.131:39097
I20260809 02:19:54.230659 10820 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.131:39097 every 8 connection(s)
I20260809 02:19:54.231493 10640 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
I20260809 02:19:54.239930 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 10640
I20260809 02:19:54.240064 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241741788589-15662-0/minicluster-data/ts-2/wal/instance
I20260809 02:19:54.240360 15662 external_mini_cluster.cc:1428] Setting key 2683f8187935af4cd92ada0b2f44fabf
I20260809 02:19:54.240496 10213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:54.233281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.131:34003 (local address 127.15.75.190:39607)
0809 02:19:54.233420 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:54.233423 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:54.234408 (+ 985us) server_negotiation.cc:407] Connection header received
0809 02:19:54.235820 (+ 1412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:54.235823 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:54.235882 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:54.235995 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:54.236894 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:54.237436 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:54.238578 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:54.238814 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:54.238928 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:54.239998 (+ 1070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:19:54.240001 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:19:54.240004 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:19:54.240016 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:19:54.240041 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:19:54.240053 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:19:54.240123 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:54.240293 (+ 170us) server_negotiation.cc:299] Negotiation successful
0809 02:19:54.240355 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 02:19:54.241499 10826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:54.233646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:39607 (local address 127.15.75.131:34003)
0809 02:19:54.234271 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:54.234310 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 02:19:54.235610 (+ 1300us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:19:54.236161 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:19:54.236168 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:19:54.236245 (+ 77us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:19:54.236745 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:54.236770 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:54.237571 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:54.237574 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:54.238450 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:54.238460 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:54.238966 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:54.238970 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:54.239136 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:54.239744 (+ 608us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:19:54.239746 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:19:54.239786 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:19:54.239844 (+ 58us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:19:54.240125 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:19:54.240137 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:19:54.240146 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 02:19:54.240292 (+ 146us) client_negotiation.cc:240] Negotiation successful
0809 02:19:54.240542 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":118,"threads_started":1}
I20260809 02:19:54.242564 10821 heartbeater.cc:347] Connected to a master server at 127.15.75.190:39607
I20260809 02:19:54.242774 10821 heartbeater.cc:464] Registering TS with master...
I20260809 02:19:54.243263 10821 heartbeater.cc:511] Master 127.15.75.190:39607 requested a full tablet report, sending...
I20260809 02:19:54.244212 10156 ts_manager.cc:194] Registered new tserver with Master: fae77d282a5f495fbd8fc9e9ab5f3278 (127.15.75.131:39097)
I20260809 02:19:54.245100 10156 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.75.131:34003
I20260809 02:19:54.254356 15662 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 02:19:54.260226 10213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:54.256392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52254 (local address 127.15.75.190:39607)
0809 02:19:54.256547 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:54.256551 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:54.256782 (+ 231us) server_negotiation.cc:407] Connection header received
0809 02:19:54.256935 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:54.256938 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:54.256981 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:54.257074 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:54.257355 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:54.257850 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:54.258714 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:54.258947 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:54.259055 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:54.259813 (+ 758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:19:54.259817 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:19:54.259820 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:19:54.259832 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:19:54.259851 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:19:54.259873 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:19:54.259951 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:54.260042 (+ 91us) server_negotiation.cc:299] Negotiation successful
0809 02:19:54.260092 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 02:19:54.263146 10156 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52254:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:19:54.266268 10156 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:19:54.280647 10846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:54.274602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.130:38099 (local address 127.0.0.1:35834)
0809 02:19:54.275490 (+ 888us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:54.275536 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 02:19:54.275661 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:19:54.276432 (+ 771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:19:54.276436 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:19:54.276455 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:19:54.276572 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:54.276578 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:54.278017 (+ 1439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:54.278021 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:54.279262 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:54.279274 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:54.280103 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:54.280106 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:54.280211 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:54.280248 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 02:19:54.280376 (+ 128us) client_negotiation.cc:240] Negotiation successful
0809 02:19:54.280489 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":710,"mutex_wait_us":54,"thread_start_us":68,"threads_started":1}
I20260809 02:19:54.280887 10851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:54.275803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47602 (local address 127.15.75.129:44415)
0809 02:19:54.276384 (+ 581us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:54.276397 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:54.276419 (+ 22us) server_negotiation.cc:407] Connection header received
0809 02:19:54.276488 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:54.276492 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:54.276672 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:54.276855 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:19:54.277257 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:54.278246 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:54.279534 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:54.280292 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:54.280395 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:54.280538 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:54.280596 (+ 58us) server_negotiation.cc:299] Negotiation successful
0809 02:19:54.280714 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":480,"thread_start_us":127,"threads_started":1}
I20260809 02:19:54.281021 10848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:54.274735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35834 (local address 127.15.75.130:38099)
0809 02:19:54.275236 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:54.275253 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:54.275822 (+ 569us) server_negotiation.cc:407] Connection header received
0809 02:19:54.275908 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:54.275916 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:54.276125 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:54.276272 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:19:54.276736 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:54.277901 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:54.279397 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:54.279975 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:54.280100 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:54.280304 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:54.280753 (+ 449us) server_negotiation.cc:299] Negotiation successful
0809 02:19:54.280869 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":150,"threads_started":1}
I20260809 02:19:54.281090 10850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:54.274683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.131:39097 (local address 127.0.0.1:57812)
0809 02:19:54.275524 (+ 841us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:54.275541 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 02:19:54.275666 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:19:54.276384 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:19:54.276390 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:19:54.276414 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:19:54.276538 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:54.276544 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:54.277712 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:54.277717 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:54.279288 (+ 1571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:54.279295 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:54.280607 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:54.280611 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:54.280752 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:54.280777 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 02:19:54.280854 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 02:19:54.280949 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":773,"mutex_wait_us":77,"thread_start_us":96,"threads_started":1}
I20260809 02:19:54.281059 10845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:54.275295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.129:44415 (local address 127.0.0.1:47602)
0809 02:19:54.275898 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:54.275911 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:19:54.275987 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:19:54.276843 (+ 856us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:19:54.276846 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:19:54.276865 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:19:54.276960 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:54.276966 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:54.278367 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:54.278372 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:54.279410 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:54.279420 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:54.280396 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:54.280400 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:54.280480 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:54.280500 (+ 20us) client_negotiation.cc:769] Sending connection context
0809 02:19:54.280559 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 02:19:54.280610 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"mutex_wait_us":3,"thread_start_us":87,"threads_started":1}
I20260809 02:19:54.281661 10849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:54.275108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57812 (local address 127.15.75.131:39097)
0809 02:19:54.275546 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:54.275560 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:54.275807 (+ 247us) server_negotiation.cc:407] Connection header received
0809 02:19:54.275892 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:54.275897 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:54.276122 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:54.276298 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:19:54.276655 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:54.277597 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:54.279417 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:54.280222 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:54.280348 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:54.280537 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:54.281370 (+ 833us) server_negotiation.cc:299] Negotiation successful
0809 02:19:54.281481 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":158,"threads_started":1}
I20260809 02:19:54.283906 10540 tablet_service.cc:1467] Processing CreateTablet for tablet 0622f5aa878843699593e0a0f5f3d940 (DEFAULT_TABLE table=test-table [id=cc5c83a71a25414ea653df2eeb05db74]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:19:54.284039 10342 tablet_service.cc:1467] Processing CreateTablet for tablet 0622f5aa878843699593e0a0f5f3d940 (DEFAULT_TABLE table=test-table [id=cc5c83a71a25414ea653df2eeb05db74]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:19:54.284192 10730 tablet_service.cc:1467] Processing CreateTablet for tablet 0622f5aa878843699593e0a0f5f3d940 (DEFAULT_TABLE table=test-table [id=cc5c83a71a25414ea653df2eeb05db74]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:19:54.285036 10540 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0622f5aa878843699593e0a0f5f3d940. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:19:54.285086 10342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0622f5aa878843699593e0a0f5f3d940. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:19:54.285086 10730 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0622f5aa878843699593e0a0f5f3d940. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:19:54.291642 10855 tablet_bootstrap.cc:492] T 0622f5aa878843699593e0a0f5f3d940 P e8e12df1d8a64b20a72071cfe0b9fc83: Bootstrap starting.
I20260809 02:19:54.292963 10856 tablet_bootstrap.cc:492] T 0622f5aa878843699593e0a0f5f3d940 P fae77d282a5f495fbd8fc9e9ab5f3278: Bootstrap starting.
I20260809 02:19:54.292963 10857 tablet_bootstrap.cc:492] T 0622f5aa878843699593e0a0f5f3d940 P 95e8d4837a734996b9df4059e4a07e72: Bootstrap starting.
I20260809 02:19:54.294054 10855 tablet_bootstrap.cc:654] T 0622f5aa878843699593e0a0f5f3d940 P e8e12df1d8a64b20a72071cfe0b9fc83: Neither blocks nor log segments found. Creating new log.
I20260809 02:19:54.294785 10855 log.cc:824] T 0622f5aa878843699593e0a0f5f3d940 P e8e12df1d8a64b20a72071cfe0b9fc83: Log is configured to *not* fsync() on all Append() calls
I20260809 02:19:54.295081 10856 tablet_bootstrap.cc:654] T 0622f5aa878843699593e0a0f5f3d940 P fae77d282a5f495fbd8fc9e9ab5f3278: Neither blocks nor log segments found. Creating new log.
I20260809 02:19:54.295081 10857 tablet_bootstrap.cc:654] T 0622f5aa878843699593e0a0f5f3d940 P 95e8d4837a734996b9df4059e4a07e72: Neither blocks nor log segments found. Creating new log.
I20260809 02:19:54.295915 10857 log.cc:824] T 0622f5aa878843699593e0a0f5f3d940 P 95e8d4837a734996b9df4059e4a07e72: Log is configured to *not* fsync() on all Append() calls
I20260809 02:19:54.295969 10856 log.cc:824] T 0622f5aa878843699593e0a0f5f3d940 P fae77d282a5f495fbd8fc9e9ab5f3278: Log is configured to *not* fsync() on all Append() calls
I20260809 02:19:54.297183 10855 tablet_bootstrap.cc:492] T 0622f5aa878843699593e0a0f5f3d940 P e8e12df1d8a64b20a72071cfe0b9fc83: No bootstrap required, opened a new log
I20260809 02:19:54.297395 10855 ts_tablet_manager.cc:1397] T 0622f5aa878843699593e0a0f5f3d940 P e8e12df1d8a64b20a72071cfe0b9fc83: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.003s
I20260809 02:19:54.297887 10857 tablet_bootstrap.cc:492] T 0622f5aa878843699593e0a0f5f3d940 P 95e8d4837a734996b9df4059e4a07e72: No bootstrap required, opened a new log
I20260809 02:19:54.297885 10856 tablet_bootstrap.cc:492] T 0622f5aa878843699593e0a0f5f3d940 P fae77d282a5f495fbd8fc9e9ab5f3278: No bootstrap required, opened a new log
I20260809 02:19:54.298071 10857 ts_tablet_manager.cc:1397] T 0622f5aa878843699593e0a0f5f3d940 P 95e8d4837a734996b9df4059e4a07e72: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 02:19:54.298071 10856 ts_tablet_manager.cc:1397] T 0622f5aa878843699593e0a0f5f3d940 P fae77d282a5f495fbd8fc9e9ab5f3278: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 02:19:54.300570 10855 raft_consensus.cc:348] T 0622f5aa878843699593e0a0f5f3d940 P e8e12df1d8a64b20a72071cfe0b9fc83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8e12df1d8a64b20a72071cfe0b9fc83" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 38099 } } peers { permanent_uuid: "fae77d282a5f495fbd8fc9e9ab5f3278" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 39097 } } peers { permanent_uuid: "95e8d4837a734996b9df4059e4a07e72" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 44415 } }
I20260809 02:19:54.300969 10855 raft_consensus.cc:374] T 0622f5aa878843699593e0a0f5f3d940 P e8e12df1d8a64b20a72071cfe0b9fc83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:19:54.301084 10855 raft_consensus.cc:729] T 0622f5aa878843699593e0a0f5f3d940 P e8e12df1d8a64b20a72071cfe0b9fc83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8e12df1d8a64b20a72071cfe0b9fc83, State: Initialized, Role: FOLLOWER
I20260809 02:19:54.301558 10855 consensus_queue.cc:260] T 0622f5aa878843699593e0a0f5f3d940 P e8e12df1d8a64b20a72071cfe0b9fc83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8e12df1d8a64b20a72071cfe0b9fc83" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 38099 } } peers { permanent_uuid: "fae77d282a5f495fbd8fc9e9ab5f3278" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 39097 } } peers { permanent_uuid: "95e8d4837a734996b9df4059e4a07e72" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 44415 } }
I20260809 02:19:54.302218 10637 heartbeater.cc:503] Master 127.15.75.190:39607 was elected leader, sending a full tablet report...
I20260809 02:19:54.302315 10857 raft_consensus.cc:348] T 0622f5aa878843699593e0a0f5f3d940 P 95e8d4837a734996b9df4059e4a07e72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8e12df1d8a64b20a72071cfe0b9fc83" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 38099 } } peers { permanent_uuid: "fae77d282a5f495fbd8fc9e9ab5f3278" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 39097 } } peers { permanent_uuid: "95e8d4837a734996b9df4059e4a07e72" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 44415 } }
I20260809 02:19:54.302291 10856 raft_consensus.cc:348] T 0622f5aa878843699593e0a0f5f3d940 P fae77d282a5f495fbd8fc9e9ab5f3278 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8e12df1d8a64b20a72071cfe0b9fc83" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 38099 } } peers { permanent_uuid: "fae77d282a5f495fbd8fc9e9ab5f3278" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 39097 } } peers { permanent_uuid: "95e8d4837a734996b9df4059e4a07e72" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 44415 } }
I20260809 02:19:54.302552 10856 raft_consensus.cc:374] T 0622f5aa878843699593e0a0f5f3d940 P fae77d282a5f495fbd8fc9e9ab5f3278 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:19:54.302636 10857 raft_consensus.cc:374] T 0622f5aa878843699593e0a0f5f3d940 P 95e8d4837a734996b9df4059e4a07e72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:19:54.302649 10856 raft_consensus.cc:729] T 0622f5aa878843699593e0a0f5f3d940 P fae77d282a5f495fbd8fc9e9ab5f3278 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fae77d282a5f495fbd8fc9e9ab5f3278, State: Initialized, Role: FOLLOWER
I20260809 02:19:54.302858 10857 raft_consensus.cc:729] T 0622f5aa878843699593e0a0f5f3d940 P 95e8d4837a734996b9df4059e4a07e72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95e8d4837a734996b9df4059e4a07e72, State: Initialized, Role: FOLLOWER
I20260809 02:19:54.303117 10856 consensus_queue.cc:260] T 0622f5aa878843699593e0a0f5f3d940 P fae77d282a5f495fbd8fc9e9ab5f3278 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8e12df1d8a64b20a72071cfe0b9fc83" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 38099 } } peers { permanent_uuid: "fae77d282a5f495fbd8fc9e9ab5f3278" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 39097 } } peers { permanent_uuid: "95e8d4837a734996b9df4059e4a07e72" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 44415 } }
I20260809 02:19:54.303333 10857 consensus_queue.cc:260] T 0622f5aa878843699593e0a0f5f3d940 P 95e8d4837a734996b9df4059e4a07e72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8e12df1d8a64b20a72071cfe0b9fc83" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 38099 } } peers { permanent_uuid: "fae77d282a5f495fbd8fc9e9ab5f3278" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 39097 } } peers { permanent_uuid: "95e8d4837a734996b9df4059e4a07e72" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 44415 } }
I20260809 02:19:54.303997 10855 ts_tablet_manager.cc:1428] T 0622f5aa878843699593e0a0f5f3d940 P e8e12df1d8a64b20a72071cfe0b9fc83: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260809 02:19:54.304075 10856 ts_tablet_manager.cc:1428] T 0622f5aa878843699593e0a0f5f3d940 P fae77d282a5f495fbd8fc9e9ab5f3278: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260809 02:19:54.304215 10857 ts_tablet_manager.cc:1428] T 0622f5aa878843699593e0a0f5f3d940 P 95e8d4837a734996b9df4059e4a07e72: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260809 02:19:54.304423 10821 heartbeater.cc:503] Master 127.15.75.190:39607 was elected leader, sending a full tablet report...
I20260809 02:19:54.304759 10442 heartbeater.cc:503] Master 127.15.75.190:39607 was elected leader, sending a full tablet report...
W20260809 02:19:54.348040 10443 tablet.cc:2369] T 0622f5aa878843699593e0a0f5f3d940 P 95e8d4837a734996b9df4059e4a07e72: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:19:54.352998 10866 raft_consensus.cc:482] T 0622f5aa878843699593e0a0f5f3d940 P 95e8d4837a734996b9df4059e4a07e72 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:19:54.353209 10866 raft_consensus.cc:504] T 0622f5aa878843699593e0a0f5f3d940 P 95e8d4837a734996b9df4059e4a07e72 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8e12df1d8a64b20a72071cfe0b9fc83" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 38099 } } peers { permanent_uuid: "fae77d282a5f495fbd8fc9e9ab5f3278" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 39097 } } peers { permanent_uuid: "95e8d4837a734996b9df4059e4a07e72" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 44415 } }
I20260809 02:19:54.354373 10866 leader_election.cc:288] T 0622f5aa878843699593e0a0f5f3d940 P 95e8d4837a734996b9df4059e4a07e72 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8e12df1d8a64b20a72071cfe0b9fc83 (127.15.75.130:38099), fae77d282a5f495fbd8fc9e9ab5f3278 (127.15.75.131:39097)
I20260809 02:19:54.358083 10444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:54.354607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.130:38099 (local address 127.15.75.129:53233)
0809 02:19:54.354752 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:54.354770 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 02:19:54.354862 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:19:54.355158 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:19:54.355161 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:19:54.355180 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:19:54.355312 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:54.355318 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:54.356140 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:54.356144 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:54.357006 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:54.357015 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:54.357667 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:54.357670 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:54.357778 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:54.357798 (+ 20us) client_negotiation.cc:769] Sending connection context
0809 02:19:54.357886 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 02:19:54.357956 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260809 02:19:54.358083 10848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:54.354732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.129:53233 (local address 127.15.75.130:38099)
0809 02:19:54.354898 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:54.354904 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:54.354936 (+ 32us) server_negotiation.cc:407] Connection header received
0809 02:19:54.354995 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:54.355001 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:54.355055 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:54.355141 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:19:54.355436 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:54.355975 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:54.357134 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:54.357537 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:54.357672 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:54.357722 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:54.357902 (+ 180us) server_negotiation.cc:299] Negotiation successful
0809 02:19:54.357964 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 02:19:54.358479 10849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:54.354764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.129:55439 (local address 127.15.75.131:39097)
0809 02:19:54.354909 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:54.354914 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:54.355220 (+ 306us) server_negotiation.cc:407] Connection header received
0809 02:19:54.355377 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:54.355385 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:54.355444 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:54.355528 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:19:54.355807 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:54.356407 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:54.357485 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:54.357862 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:54.358003 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:54.358049 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:54.358279 (+ 230us) server_negotiation.cc:299] Negotiation successful
0809 02:19:54.358369 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 02:19:54.358495 10867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:54.354622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.131:39097 (local address 127.15.75.129:55439)
0809 02:19:54.355113 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:54.355125 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:19:54.355254 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:19:54.355541 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:19:54.355544 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:19:54.355560 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:19:54.355694 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:54.355700 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:54.356525 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:54.356529 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:54.357342 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:54.357351 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:54.358003 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:54.358007 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:54.358111 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:54.358139 (+ 28us) client_negotiation.cc:769] Sending connection context
0809 02:19:54.358246 (+ 107us) client_negotiation.cc:240] Negotiation successful
0809 02:19:54.358324 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":433,"mutex_wait_us":113,"thread_start_us":105,"threads_started":1}
I20260809 02:19:54.358855 10568 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0622f5aa878843699593e0a0f5f3d940" candidate_uuid: "95e8d4837a734996b9df4059e4a07e72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8e12df1d8a64b20a72071cfe0b9fc83" is_pre_election: true
I20260809 02:19:54.359155 10568 raft_consensus.cc:2454] T 0622f5aa878843699593e0a0f5f3d940 P e8e12df1d8a64b20a72071cfe0b9fc83 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95e8d4837a734996b9df4059e4a07e72 in term 0.
I20260809 02:19:54.359184 10759 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0622f5aa878843699593e0a0f5f3d940" candidate_uuid: "95e8d4837a734996b9df4059e4a07e72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fae77d282a5f495fbd8fc9e9ab5f3278" is_pre_election: true
I20260809 02:19:54.359516 10759 raft_consensus.cc:2454] T 0622f5aa878843699593e0a0f5f3d940 P fae77d282a5f495fbd8fc9e9ab5f3278 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95e8d4837a734996b9df4059e4a07e72 in term 0.
I20260809 02:19:54.359719 10275 leader_election.cc:302] T 0622f5aa878843699593e0a0f5f3d940 P 95e8d4837a734996b9df4059e4a07e72 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95e8d4837a734996b9df4059e4a07e72, e8e12df1d8a64b20a72071cfe0b9fc83; no voters:
I20260809 02:19:54.359993 10866 raft_consensus.cc:2781] T 0622f5aa878843699593e0a0f5f3d940 P 95e8d4837a734996b9df4059e4a07e72 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:19:54.360085 10866 raft_consensus.cc:482] T 0622f5aa878843699593e0a0f5f3d940 P 95e8d4837a734996b9df4059e4a07e72 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:19:54.360128 10866 raft_consensus.cc:3037] T 0622f5aa878843699593e0a0f5f3d940 P 95e8d4837a734996b9df4059e4a07e72 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:19:54.361167 10866 raft_consensus.cc:504] T 0622f5aa878843699593e0a0f5f3d940 P 95e8d4837a734996b9df4059e4a07e72 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8e12df1d8a64b20a72071cfe0b9fc83" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 38099 } } peers { permanent_uuid: "fae77d282a5f495fbd8fc9e9ab5f3278" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 39097 } } peers { permanent_uuid: "95e8d4837a734996b9df4059e4a07e72" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 44415 } }
I20260809 02:19:54.361518 10866 leader_election.cc:288] T 0622f5aa878843699593e0a0f5f3d940 P 95e8d4837a734996b9df4059e4a07e72 [CANDIDATE]: Term 1 election: Requested vote from peers e8e12df1d8a64b20a72071cfe0b9fc83 (127.15.75.130:38099), fae77d282a5f495fbd8fc9e9ab5f3278 (127.15.75.131:39097)
I20260809 02:19:54.361807 10568 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0622f5aa878843699593e0a0f5f3d940" candidate_uuid: "95e8d4837a734996b9df4059e4a07e72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8e12df1d8a64b20a72071cfe0b9fc83"
I20260809 02:19:54.361943 10568 raft_consensus.cc:3037] T 0622f5aa878843699593e0a0f5f3d940 P e8e12df1d8a64b20a72071cfe0b9fc83 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:19:54.361943 10759 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0622f5aa878843699593e0a0f5f3d940" candidate_uuid: "95e8d4837a734996b9df4059e4a07e72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fae77d282a5f495fbd8fc9e9ab5f3278"
I20260809 02:19:54.362087 10759 raft_consensus.cc:3037] T 0622f5aa878843699593e0a0f5f3d940 P fae77d282a5f495fbd8fc9e9ab5f3278 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:19:54.363041 10568 raft_consensus.cc:2454] T 0622f5aa878843699593e0a0f5f3d940 P e8e12df1d8a64b20a72071cfe0b9fc83 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95e8d4837a734996b9df4059e4a07e72 in term 1.
I20260809 02:19:54.363103 10759 raft_consensus.cc:2454] T 0622f5aa878843699593e0a0f5f3d940 P fae77d282a5f495fbd8fc9e9ab5f3278 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95e8d4837a734996b9df4059e4a07e72 in term 1.
I20260809 02:19:54.363350 10275 leader_election.cc:302] T 0622f5aa878843699593e0a0f5f3d940 P 95e8d4837a734996b9df4059e4a07e72 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95e8d4837a734996b9df4059e4a07e72, e8e12df1d8a64b20a72071cfe0b9fc83; no voters:
I20260809 02:19:54.363509 10866 raft_consensus.cc:2781] T 0622f5aa878843699593e0a0f5f3d940 P 95e8d4837a734996b9df4059e4a07e72 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:19:54.363739 10866 raft_consensus.cc:686] T 0622f5aa878843699593e0a0f5f3d940 P 95e8d4837a734996b9df4059e4a07e72 [term 1 LEADER]: Becoming Leader. State: Replica: 95e8d4837a734996b9df4059e4a07e72, State: Running, Role: LEADER
I20260809 02:19:54.364063 10866 consensus_queue.cc:237] T 0622f5aa878843699593e0a0f5f3d940 P 95e8d4837a734996b9df4059e4a07e72 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8e12df1d8a64b20a72071cfe0b9fc83" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 38099 } } peers { permanent_uuid: "fae77d282a5f495fbd8fc9e9ab5f3278" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 39097 } } peers { permanent_uuid: "95e8d4837a734996b9df4059e4a07e72" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 44415 } }
I20260809 02:19:54.367383 10155 catalog_manager.cc:5495] T 0622f5aa878843699593e0a0f5f3d940 P 95e8d4837a734996b9df4059e4a07e72 reported cstate change: term changed from 0 to 1, leader changed from <none> to 95e8d4837a734996b9df4059e4a07e72 (127.15.75.129). New cstate: current_term: 1 leader_uuid: "95e8d4837a734996b9df4059e4a07e72" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8e12df1d8a64b20a72071cfe0b9fc83" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 38099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fae77d282a5f495fbd8fc9e9ab5f3278" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 39097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95e8d4837a734996b9df4059e4a07e72" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 44415 } health_report { overall_health: HEALTHY } } }
I20260809 02:19:54.401840 10851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:54.398540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49212 (local address 127.15.75.129:44415)
0809 02:19:54.398676 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:54.398680 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:54.398692 (+ 12us) server_negotiation.cc:407] Connection header received
0809 02:19:54.398770 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:54.398772 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:54.398826 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:54.398920 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:19:54.399152 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:54.399654 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:54.400484 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:54.400741 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:54.400839 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:54.401109 (+ 270us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:19:54.401512 (+ 403us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:19:54.401605 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:54.401666 (+ 61us) server_negotiation.cc:299] Negotiation successful
0809 02:19:54.401711 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 02:19:54.407717 10759 raft_consensus.cc:1261] T 0622f5aa878843699593e0a0f5f3d940 P fae77d282a5f495fbd8fc9e9ab5f3278 [term 1 FOLLOWER]: Refusing update from remote peer 95e8d4837a734996b9df4059e4a07e72: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:19:54.407753 10568 raft_consensus.cc:1261] T 0622f5aa878843699593e0a0f5f3d940 P e8e12df1d8a64b20a72071cfe0b9fc83 [term 1 FOLLOWER]: Refusing update from remote peer 95e8d4837a734996b9df4059e4a07e72: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:19:54.408452 10868 consensus_queue.cc:1035] T 0622f5aa878843699593e0a0f5f3d940 P 95e8d4837a734996b9df4059e4a07e72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8e12df1d8a64b20a72071cfe0b9fc83" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 38099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:19:54.408716 10866 consensus_queue.cc:1035] T 0622f5aa878843699593e0a0f5f3d940 P 95e8d4837a734996b9df4059e4a07e72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fae77d282a5f495fbd8fc9e9ab5f3278" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 39097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:19:54.415587 10871 mvcc.cc:205] Tried to move back new op lower bound from 7316447209086480384 to 7316447208917917696. Current Snapshot: MvccSnapshot[applied={T|T < 7316447209086480384}]
I20260809 02:19:54.417699 10872 mvcc.cc:205] Tried to move back new op lower bound from 7316447209086480384 to 7316447208917917696. Current Snapshot: MvccSnapshot[applied={T|T < 7316447209086480384}]
I20260809 02:19:54.418100 10873 mvcc.cc:205] Tried to move back new op lower bound from 7316447209086480384 to 7316447208917917696. Current Snapshot: MvccSnapshot[applied={T|T < 7316447209086480384}]
I20260809 02:19:54.424101 10155 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:52254:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:19:54.424274 10155 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:52254:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:19:54.426630 10155 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1a8c954996b24d0e856a6c63fd7a02f6: Sending DeleteTablet for 3 replicas of tablet 0622f5aa878843699593e0a0f5f3d940
I20260809 02:19:54.427445 10730 tablet_service.cc:1514] Processing DeleteTablet for tablet 0622f5aa878843699593e0a0f5f3d940 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:19:54 UTC) from {username='slave'} at 127.0.0.1:57812
I20260809 02:19:54.427488 10342 tablet_service.cc:1514] Processing DeleteTablet for tablet 0622f5aa878843699593e0a0f5f3d940 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:19:54 UTC) from {username='slave'} at 127.0.0.1:47602
I20260809 02:19:54.427969 10883 tablet_replica.cc:330] T 0622f5aa878843699593e0a0f5f3d940 P fae77d282a5f495fbd8fc9e9ab5f3278: stopping tablet replica
I20260809 02:19:54.428001 10540 tablet_service.cc:1514] Processing DeleteTablet for tablet 0622f5aa878843699593e0a0f5f3d940 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:19:54 UTC) from {username='slave'} at 127.0.0.1:35834
I20260809 02:19:54.428344 10883 raft_consensus.cc:2229] T 0622f5aa878843699593e0a0f5f3d940 P fae77d282a5f495fbd8fc9e9ab5f3278 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:19:54.428514 10885 tablet_replica.cc:330] T 0622f5aa878843699593e0a0f5f3d940 P e8e12df1d8a64b20a72071cfe0b9fc83: stopping tablet replica
I20260809 02:19:54.428617 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 10248
I20260809 02:19:54.429015 10885 raft_consensus.cc:2229] T 0622f5aa878843699593e0a0f5f3d940 P e8e12df1d8a64b20a72071cfe0b9fc83 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:19:54.429373 10885 raft_consensus.cc:2258] T 0622f5aa878843699593e0a0f5f3d940 P e8e12df1d8a64b20a72071cfe0b9fc83 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:19:54.444427 10883 raft_consensus.cc:2258] T 0622f5aa878843699593e0a0f5f3d940 P fae77d282a5f495fbd8fc9e9ab5f3278 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:19:54.445140 10885 ts_tablet_manager.cc:1905] T 0622f5aa878843699593e0a0f5f3d940 P e8e12df1d8a64b20a72071cfe0b9fc83: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:19:54.446069 10883 ts_tablet_manager.cc:1905] T 0622f5aa878843699593e0a0f5f3d940 P fae77d282a5f495fbd8fc9e9ab5f3278: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:19:54.447507 10885 ts_tablet_manager.cc:1918] T 0622f5aa878843699593e0a0f5f3d940 P e8e12df1d8a64b20a72071cfe0b9fc83: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:19:54.447628 10885 log.cc:1196] T 0622f5aa878843699593e0a0f5f3d940 P e8e12df1d8a64b20a72071cfe0b9fc83: Deleting WAL directory at /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241741788589-15662-0/minicluster-data/ts-1/wal/wals/0622f5aa878843699593e0a0f5f3d940
I20260809 02:19:54.447957 10885 ts_tablet_manager.cc:1939] T 0622f5aa878843699593e0a0f5f3d940 P e8e12df1d8a64b20a72071cfe0b9fc83: Deleting consensus metadata
I20260809 02:19:54.449172 10137 catalog_manager.cc:4841] TS e8e12df1d8a64b20a72071cfe0b9fc83 (127.15.75.130:38099): tablet 0622f5aa878843699593e0a0f5f3d940 (table test-table [id=cc5c83a71a25414ea653df2eeb05db74]) successfully deleted
I20260809 02:19:54.449600 10883 ts_tablet_manager.cc:1918] T 0622f5aa878843699593e0a0f5f3d940 P fae77d282a5f495fbd8fc9e9ab5f3278: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:19:54.449708 10883 log.cc:1196] T 0622f5aa878843699593e0a0f5f3d940 P fae77d282a5f495fbd8fc9e9ab5f3278: Deleting WAL directory at /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241741788589-15662-0/minicluster-data/ts-2/wal/wals/0622f5aa878843699593e0a0f5f3d940
I20260809 02:19:54.450029 10883 ts_tablet_manager.cc:1939] T 0622f5aa878843699593e0a0f5f3d940 P fae77d282a5f495fbd8fc9e9ab5f3278: Deleting consensus metadata
I20260809 02:19:54.451335 10134 catalog_manager.cc:4841] TS fae77d282a5f495fbd8fc9e9ab5f3278 (127.15.75.131:39097): tablet 0622f5aa878843699593e0a0f5f3d940 (table test-table [id=cc5c83a71a25414ea653df2eeb05db74]) successfully deleted
W20260809 02:19:54.451947 10137 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.75.129:44415 (error 108)
I20260809 02:19:54.452147 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 10445
I20260809 02:19:54.459169 10850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:54.456277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.129:44415 (local address 127.0.0.1:49218)
0809 02:19:54.459026 (+ 2749us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:54.459095 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.129:44415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2621}
W20260809 02:19:54.459991 10137 catalog_manager.cc:4584] TS 95e8d4837a734996b9df4059e4a07e72 (127.15.75.129:44415): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0622f5aa878843699593e0a0f5f3d940: Network error: Client connection negotiation failed: client connection to 127.15.75.129:44415: connect: Connection refused (error 111)
I20260809 02:19:54.466126 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 10640
I20260809 02:19:54.478377 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 10114
2026-08-09T02:19:54Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1121 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:19:54 dist-test-slave-s11t krb5kdc[10892](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:19:54 dist-test-slave-s11t krb5kdc[10892](info): set up 2 sockets
Aug 09 02:19:54 dist-test-slave-s11t krb5kdc[10892](info): commencing operation
krb5kdc: starting...
W20260809 02:19:55.710402 15662 mini_kdc.cc:121] Time spent starting KDC: real 1.205s user 0.005s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Aug 09 02:19:55 dist-test-slave-s11t krb5kdc[10892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241995, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:19:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:19:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.75.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab.
Entry for principal kudu/127.15.75.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab.
I20260809 02:19:55.832810 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:44215
--webserver_interface=127.15.75.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab
--principal=kudu/127.15.75.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:41173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.75.190:44215
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:19:55.943133 10911 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:19:55.943477 10911 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:19:55.943554 10911 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:19:55.947564 10911 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:19:55.947643 10911 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:19:55.947681 10911 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:19:55.947718 10911 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:19:55.947750 10911 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:19:55.952952 10911 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:41173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.75.190:44215
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab
--principal=kudu/127.15.75.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:44215
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.75.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10911
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:19:55.953949 10911 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:19:55.954932 10911 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:19:55.963253 10924 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:19:55.963261 10921 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:19:55.963277 10919 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:19:55.963526 10911 server_base.cc:1034] running on GCE node
I20260809 02:19:55.964157 10911 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:19:55.965211 10911 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:19:55.966375 10911 hybrid_clock.cc:648] HybridClock initialized: now 1786241995966355 us; error 31 us; skew 500 ppm
Aug 09 02:19:55 dist-test-slave-s11t krb5kdc[10892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241995, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:19:55.969105 10911 init.cc:377] Logged in from keytab as kudu/127.15.75.190@KRBTEST.COM (short username kudu)
I20260809 02:19:55.969875 10911 webserver.cc:460] Webserver started at http://127.15.75.190:36367/ using document root <none> and password file <none>
I20260809 02:19:55.970367 10911 fs_manager.cc:359] Metadata directory not provided
I20260809 02:19:55.970422 10911 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:19:55.970577 10911 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:19:55.972440 10911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/minicluster-data/master-0/data/instance:
uuid: "3c64e55e35a34e21ba907f6af834ae02"
format_stamp: "Formatted at 2026-08-09 02:19:55 on dist-test-slave-s11t"
server_key: "6419b2f5e71d544485c3048b5d01bb7a"
server_key_iv: "d5adc6e13e6b76547b6ea85a4679395b"
server_key_version: "encryptionkey@0"
I20260809 02:19:55.973008 10911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/minicluster-data/master-0/wal/instance:
uuid: "3c64e55e35a34e21ba907f6af834ae02"
format_stamp: "Formatted at 2026-08-09 02:19:55 on dist-test-slave-s11t"
server_key: "6419b2f5e71d544485c3048b5d01bb7a"
server_key_iv: "d5adc6e13e6b76547b6ea85a4679395b"
server_key_version: "encryptionkey@0"
I20260809 02:19:55.976397 10911 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.002s
I20260809 02:19:55.978714 10934 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:19:55.980039 10911 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 02:19:55.980188 10911 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/minicluster-data/master-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/minicluster-data/master-0/wal
uuid: "3c64e55e35a34e21ba907f6af834ae02"
format_stamp: "Formatted at 2026-08-09 02:19:55 on dist-test-slave-s11t"
server_key: "6419b2f5e71d544485c3048b5d01bb7a"
server_key_iv: "d5adc6e13e6b76547b6ea85a4679395b"
server_key_version: "encryptionkey@0"
I20260809 02:19:55.980301 10911 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:19:55.997260 10911 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:19:55.999995 10911 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:19:56.000207 10911 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:19:56.007516 10911 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.190:44215
I20260809 02:19:56.007545 11002 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.190:44215 every 8 connection(s)
I20260809 02:19:56.008441 10911 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
I20260809 02:19:56.011266 11004 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:19:56.016789 11004 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c64e55e35a34e21ba907f6af834ae02: Bootstrap starting.
I20260809 02:19:56.018776 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 10911
I20260809 02:19:56.018927 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/minicluster-data/master-0/wal/instance
I20260809 02:19:56.019224 15662 external_mini_cluster.cc:1428] Setting key 4e3398dfcd377e6eafe92ea1772b9150
I20260809 02:19:56.019501 11004 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3c64e55e35a34e21ba907f6af834ae02: Neither blocks nor log segments found. Creating new log.
I20260809 02:19:56.020321 11004 log.cc:824] T 00000000000000000000000000000000 P 3c64e55e35a34e21ba907f6af834ae02: Log is configured to *not* fsync() on all Append() calls
I20260809 02:19:56.022332 11004 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c64e55e35a34e21ba907f6af834ae02: No bootstrap required, opened a new log
I20260809 02:19:56.026046 11004 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3c64e55e35a34e21ba907f6af834ae02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c64e55e35a34e21ba907f6af834ae02" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 44215 } }
I20260809 02:19:56.026325 11004 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3c64e55e35a34e21ba907f6af834ae02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:19:56.026424 11004 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3c64e55e35a34e21ba907f6af834ae02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c64e55e35a34e21ba907f6af834ae02, State: Initialized, Role: FOLLOWER
I20260809 02:19:56.026933 11004 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3c64e55e35a34e21ba907f6af834ae02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c64e55e35a34e21ba907f6af834ae02" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 44215 } }
I20260809 02:19:56.027056 11004 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3c64e55e35a34e21ba907f6af834ae02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:19:56.027124 11004 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3c64e55e35a34e21ba907f6af834ae02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:19:56.027241 11004 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3c64e55e35a34e21ba907f6af834ae02 [term 0 FOLLOWER]: Advancing to term 1
Aug 09 02:19:56 dist-test-slave-s11t krb5kdc[10892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241995, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:19:56.028093 11004 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3c64e55e35a34e21ba907f6af834ae02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c64e55e35a34e21ba907f6af834ae02" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 44215 } }
I20260809 02:19:56.028451 11004 leader_election.cc:302] T 00000000000000000000000000000000 P 3c64e55e35a34e21ba907f6af834ae02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3c64e55e35a34e21ba907f6af834ae02; no voters:
I20260809 02:19:56.028805 11004 leader_election.cc:288] T 00000000000000000000000000000000 P 3c64e55e35a34e21ba907f6af834ae02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:19:56.028872 11014 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3c64e55e35a34e21ba907f6af834ae02 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:19:56.029089 11014 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3c64e55e35a34e21ba907f6af834ae02 [term 1 LEADER]: Becoming Leader. State: Replica: 3c64e55e35a34e21ba907f6af834ae02, State: Running, Role: LEADER
I20260809 02:19:56.029714 11014 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3c64e55e35a34e21ba907f6af834ae02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c64e55e35a34e21ba907f6af834ae02" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 44215 } }
I20260809 02:19:56.030114 11004 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3c64e55e35a34e21ba907f6af834ae02 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:19:56.031412 11015 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c64e55e35a34e21ba907f6af834ae02 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3c64e55e35a34e21ba907f6af834ae02. Latest consensus state: current_term: 1 leader_uuid: "3c64e55e35a34e21ba907f6af834ae02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c64e55e35a34e21ba907f6af834ae02" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 44215 } } }
I20260809 02:19:56.031570 11015 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c64e55e35a34e21ba907f6af834ae02 [sys.catalog]: This master's current role is: LEADER
I20260809 02:19:56.031930 11017 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c64e55e35a34e21ba907f6af834ae02 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3c64e55e35a34e21ba907f6af834ae02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c64e55e35a34e21ba907f6af834ae02" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 44215 } } }
I20260809 02:19:56.032068 11017 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c64e55e35a34e21ba907f6af834ae02 [sys.catalog]: This master's current role is: LEADER
I20260809 02:19:56.034612 11008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:56.020402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51046 (local address 127.15.75.190:44215)
0809 02:19:56.020884 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:56.020908 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:56.020955 (+ 47us) server_negotiation.cc:407] Connection header received
0809 02:19:56.021643 (+ 688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:56.021661 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:56.021973 (+ 312us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:56.022316 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:56.023155 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:56.024258 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:56.025217 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:56.025686 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:56.025859 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:56.028756 (+ 2897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:19:56.028767 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:19:56.028780 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:19:56.028808 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:19:56.030893 (+ 2085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:56.031510 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:56.031515 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:56.031521 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:56.031667 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:56.032045 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:56.032048 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:56.032049 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:56.032710 (+ 661us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:19:56.032916 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:56.033219 (+ 303us) server_negotiation.cc:299] Negotiation successful
0809 02:19:56.033685 (+ 466us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":127,"threads_started":1}
W20260809 02:19:56.035218 11037 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3c64e55e35a34e21ba907f6af834ae02: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:19:56.035296 11037 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:19:56.035355 11034 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:19:56.036195 11034 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:19:56.041409 11034 catalog_manager.cc:1347] Generated new cluster ID: 5223a523d27245fda6e51d67679f39a4
I20260809 02:19:56.041468 11034 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:19:56.049019 11034 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:19:56.049846 11034 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:19:56.056914 11034 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3c64e55e35a34e21ba907f6af834ae02: Generated new TSK 0
I20260809 02:19:56.057571 11034 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.75.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab.
Entry for principal kudu/127.15.75.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab.
I20260809 02:19:56.089501 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.129:0
--local_ip_for_outbound_sockets=127.15.75.129
--webserver_interface=127.15.75.129
--webserver_port=0
--tserver_master_addrs=127.15.75.190:44215
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab
--principal=kudu/127.15.75.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:41173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:19:56.205077 11047 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:19:56.205317 11047 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:19:56.205396 11047 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:19:56.209378 11047 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:19:56.209429 11047 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:19:56.209529 11047 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.129
I20260809 02:19:56.214654 11047 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:41173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab
--principal=kudu/127.15.75.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.75.129
--webserver_port=0
--tserver_master_addrs=127.15.75.190:44215
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11047
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.129
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:19:56.215688 11047 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:19:56.216666 11047 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:19:56.226251 11056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:19:56.226260 11059 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:19:56.226403 11047 server_base.cc:1034] running on GCE node
W20260809 02:19:56.226614 11063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:19:56.226992 11047 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:19:56.227599 11047 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:19:56.228785 11047 hybrid_clock.cc:648] HybridClock initialized: now 1786241996228768 us; error 25 us; skew 500 ppm
Aug 09 02:19:56 dist-test-slave-s11t krb5kdc[10892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241996, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:19:56.231701 11047 init.cc:377] Logged in from keytab as kudu/127.15.75.129@KRBTEST.COM (short username kudu)
I20260809 02:19:56.232576 11047 webserver.cc:460] Webserver started at http://127.15.75.129:38879/ using document root <none> and password file <none>
I20260809 02:19:56.233139 11047 fs_manager.cc:359] Metadata directory not provided
I20260809 02:19:56.233230 11047 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:19:56.233464 11047 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:19:56.235390 11047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/minicluster-data/ts-0/data/instance:
uuid: "b3c787537d53481c8846ea80108c1e65"
format_stamp: "Formatted at 2026-08-09 02:19:56 on dist-test-slave-s11t"
server_key: "59edda10ea61904345ea363d2595a61d"
server_key_iv: "0e8d5a61db2d062013be443374ae2a57"
server_key_version: "encryptionkey@0"
I20260809 02:19:56.235975 11047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/minicluster-data/ts-0/wal/instance:
uuid: "b3c787537d53481c8846ea80108c1e65"
format_stamp: "Formatted at 2026-08-09 02:19:56 on dist-test-slave-s11t"
server_key: "59edda10ea61904345ea363d2595a61d"
server_key_iv: "0e8d5a61db2d062013be443374ae2a57"
server_key_version: "encryptionkey@0"
I20260809 02:19:56.239500 11047 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260809 02:19:56.241909 11072 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:19:56.243067 11047 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 02:19:56.243217 11047 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/minicluster-data/ts-0/wal
uuid: "b3c787537d53481c8846ea80108c1e65"
format_stamp: "Formatted at 2026-08-09 02:19:56 on dist-test-slave-s11t"
server_key: "59edda10ea61904345ea363d2595a61d"
server_key_iv: "0e8d5a61db2d062013be443374ae2a57"
server_key_version: "encryptionkey@0"
I20260809 02:19:56.243323 11047 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:19:56.263005 11047 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:19:56.266378 11047 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:19:56.266605 11047 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:19:56.267220 11047 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:19:56.268040 11047 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:19:56.268114 11047 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:19:56.268184 11047 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:19:56.268239 11047 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:19:56.277942 11047 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.129:43373
I20260809 02:19:56.277968 11245 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.129:43373 every 8 connection(s)
I20260809 02:19:56.278782 11047 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
I20260809 02:19:56.286293 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 11047
I20260809 02:19:56.286443 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/minicluster-data/ts-0/wal/instance
I20260809 02:19:56.286760 15662 external_mini_cluster.cc:1428] Setting key 73c7f03ac04bba696fc01c170fbf8c37
Aug 09 02:19:56 dist-test-slave-s11t krb5kdc[10892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241996, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.129@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:19:56.292531 11008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:56.280527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.129:40423 (local address 127.15.75.190:44215)
0809 02:19:56.280710 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:56.280716 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:56.281538 (+ 822us) server_negotiation.cc:407] Connection header received
0809 02:19:56.282795 (+ 1257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:56.282798 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:56.282845 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:56.282923 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:56.284208 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:56.284838 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:56.285906 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:56.286231 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:56.286353 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:56.289107 (+ 2754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:19:56.289110 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:19:56.289112 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:19:56.289134 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:19:56.290479 (+ 1345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:56.291005 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:56.291012 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:56.291014 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:56.291087 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:56.291432 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:56.291437 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:56.291438 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:56.291614 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:19:56.291723 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:56.292309 (+ 586us) server_negotiation.cc:299] Negotiation successful
0809 02:19:56.292416 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 02:19:56.293656 11248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:56.280907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:44215 (local address 127.15.75.129:40423)
0809 02:19:56.281380 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:56.281432 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 02:19:56.282592 (+ 1160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:19:56.283129 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:19:56.283140 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:19:56.283488 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:19:56.284016 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:56.284034 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:56.284952 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:56.284955 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:56.285806 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:56.285815 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:56.286413 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:56.286417 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:56.286624 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:56.287219 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:19:56.287239 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:19:56.288950 (+ 1711us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:19:56.290604 (+ 1654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:19:56.290609 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:19:56.290622 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:19:56.290875 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:19:56.291182 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:19:56.291185 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:19:56.291187 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:19:56.291329 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:19:56.291723 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:19:56.291729 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:19:56.292028 (+ 299us) client_negotiation.cc:769] Sending connection context
0809 02:19:56.292234 (+ 206us) client_negotiation.cc:240] Negotiation successful
0809 02:19:56.292573 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":107,"threads_started":1}
I20260809 02:19:56.295131 11246 heartbeater.cc:347] Connected to a master server at 127.15.75.190:44215
I20260809 02:19:56.295400 11246 heartbeater.cc:464] Registering TS with master...
I20260809 02:19:56.296003 11246 heartbeater.cc:511] Master 127.15.75.190:44215 requested a full tablet report, sending...
I20260809 02:19:56.297595 10954 ts_manager.cc:194] Registered new tserver with Master: b3c787537d53481c8846ea80108c1e65 (127.15.75.129:43373)
WARNING: no policy specified for kudu/127.15.75.130@KRBTEST.COM; defaulting to no policy
I20260809 02:19:56.298764 10954 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.75.129@KRBTEST.COM'} at 127.15.75.129:40423
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab.
Entry for principal kudu/127.15.75.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab.
I20260809 02:19:56.317325 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.130:0
--local_ip_for_outbound_sockets=127.15.75.130
--webserver_interface=127.15.75.130
--webserver_port=0
--tserver_master_addrs=127.15.75.190:44215
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab
--principal=kudu/127.15.75.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:41173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:19:56.427879 11251 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:19:56.428107 11251 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:19:56.428172 11251 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:19:56.432379 11251 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:19:56.432432 11251 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:19:56.432488 11251 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.130
I20260809 02:19:56.437640 11251 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:41173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab
--principal=kudu/127.15.75.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.75.130
--webserver_port=0
--tserver_master_addrs=127.15.75.190:44215
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11251
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.130
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:19:56.438565 11251 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:19:56.439515 11251 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:19:56.448621 11262 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:19:56.448688 11265 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:19:56.448696 11259 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:19:56.448956 11251 server_base.cc:1034] running on GCE node
I20260809 02:19:56.449384 11251 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:19:56.449968 11251 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:19:56.451169 11251 hybrid_clock.cc:648] HybridClock initialized: now 1786241996451159 us; error 34 us; skew 500 ppm
Aug 09 02:19:56 dist-test-slave-s11t krb5kdc[10892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241996, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:19:56.453856 11251 init.cc:377] Logged in from keytab as kudu/127.15.75.130@KRBTEST.COM (short username kudu)
I20260809 02:19:56.454648 11251 webserver.cc:460] Webserver started at http://127.15.75.130:45987/ using document root <none> and password file <none>
I20260809 02:19:56.455180 11251 fs_manager.cc:359] Metadata directory not provided
I20260809 02:19:56.455269 11251 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:19:56.455493 11251 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:19:56.457485 11251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/minicluster-data/ts-1/data/instance:
uuid: "0838a1e5d0234d0da692fd177343c2eb"
format_stamp: "Formatted at 2026-08-09 02:19:56 on dist-test-slave-s11t"
server_key: "68fb6b820b73abc5bfeac2572e990a86"
server_key_iv: "7be7f8c4de40e99cbeae7e4bcc9c0146"
server_key_version: "encryptionkey@0"
I20260809 02:19:56.458038 11251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/minicluster-data/ts-1/wal/instance:
uuid: "0838a1e5d0234d0da692fd177343c2eb"
format_stamp: "Formatted at 2026-08-09 02:19:56 on dist-test-slave-s11t"
server_key: "68fb6b820b73abc5bfeac2572e990a86"
server_key_iv: "7be7f8c4de40e99cbeae7e4bcc9c0146"
server_key_version: "encryptionkey@0"
I20260809 02:19:56.461609 11251 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 02:19:56.463886 11279 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:19:56.465091 11251 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 02:19:56.465217 11251 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/minicluster-data/ts-1/wal
uuid: "0838a1e5d0234d0da692fd177343c2eb"
format_stamp: "Formatted at 2026-08-09 02:19:56 on dist-test-slave-s11t"
server_key: "68fb6b820b73abc5bfeac2572e990a86"
server_key_iv: "7be7f8c4de40e99cbeae7e4bcc9c0146"
server_key_version: "encryptionkey@0"
I20260809 02:19:56.465320 11251 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:19:56.475867 11251 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:19:56.478740 11251 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:19:56.478946 11251 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:19:56.479497 11251 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:19:56.480297 11251 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:19:56.480368 11251 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:19:56.480435 11251 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:19:56.480479 11251 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:19:56.489480 11251 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.130:35883
I20260809 02:19:56.489506 11446 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.130:35883 every 8 connection(s)
I20260809 02:19:56.490329 11251 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
I20260809 02:19:56.493345 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 11251
I20260809 02:19:56.493498 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/minicluster-data/ts-1/wal/instance
I20260809 02:19:56.493883 15662 external_mini_cluster.cc:1428] Setting key 42d141a8215981ef95c0e87d04b320ac
Aug 09 02:19:56 dist-test-slave-s11t krb5kdc[10892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241996, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.130@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.75.131@KRBTEST.COM; defaulting to no policy
I20260809 02:19:56.505877 11008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:56.492022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.130:58213 (local address 127.15.75.190:44215)
0809 02:19:56.492160 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:56.492164 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:56.492959 (+ 795us) server_negotiation.cc:407] Connection header received
0809 02:19:56.495131 (+ 2172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:56.495133 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:56.495167 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:56.495262 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:56.496497 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:56.497093 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:56.498908 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:56.499137 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:56.499231 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:56.502113 (+ 2882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:19:56.502116 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:19:56.502119 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:19:56.502144 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:19:56.503693 (+ 1549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:56.504261 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:56.504264 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:56.504265 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:56.504339 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:56.504713 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:56.504717 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:56.504719 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:56.504899 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:19:56.505018 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:56.505595 (+ 577us) server_negotiation.cc:299] Negotiation successful
0809 02:19:56.505717 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 02:19:56.506573 11450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:56.492327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:44215 (local address 127.15.75.130:58213)
0809 02:19:56.492819 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:56.492860 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 02:19:56.494477 (+ 1617us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:19:56.495507 (+ 1030us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:19:56.495514 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:19:56.495861 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:19:56.496349 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:56.496362 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:56.497217 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:56.497220 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:56.498760 (+ 1540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:56.498772 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:56.499295 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:56.499302 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:56.499538 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:56.500149 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:19:56.500173 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:19:56.501915 (+ 1742us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:19:56.503832 (+ 1917us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:19:56.503838 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:19:56.503851 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:19:56.504124 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:19:56.504451 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:19:56.504454 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:19:56.504456 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:19:56.504596 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:19:56.505039 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:19:56.505046 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:19:56.505345 (+ 299us) client_negotiation.cc:769] Sending connection context
0809 02:19:56.505503 (+ 158us) client_negotiation.cc:240] Negotiation successful
0809 02:19:56.505755 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":112,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.131@KRBTEST.COM" created.
I20260809 02:19:56.507824 11447 heartbeater.cc:347] Connected to a master server at 127.15.75.190:44215
I20260809 02:19:56.508100 11447 heartbeater.cc:464] Registering TS with master...
I20260809 02:19:56.508677 11447 heartbeater.cc:511] Master 127.15.75.190:44215 requested a full tablet report, sending...
I20260809 02:19:56.509833 10954 ts_manager.cc:194] Registered new tserver with Master: 0838a1e5d0234d0da692fd177343c2eb (127.15.75.130:35883)
I20260809 02:19:56.510390 10954 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.75.130@KRBTEST.COM'} at 127.15.75.130:58213
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab.
Entry for principal kudu/127.15.75.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab.
I20260809 02:19:56.523321 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.131:0
--local_ip_for_outbound_sockets=127.15.75.131
--webserver_interface=127.15.75.131
--webserver_port=0
--tserver_master_addrs=127.15.75.190:44215
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab
--principal=kudu/127.15.75.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:41173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:19:56.633208 11453 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:19:56.633469 11453 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:19:56.633549 11453 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:19:56.637521 11453 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:19:56.637593 11453 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:19:56.637676 11453 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.131
I20260809 02:19:56.642983 11453 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:41173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab
--principal=kudu/127.15.75.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.75.131
--webserver_port=0
--tserver_master_addrs=127.15.75.190:44215
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11453
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.131
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:19:56.643991 11453 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:19:56.645076 11453 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:19:56.654510 11461 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:19:56.654609 11459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:19:56.654526 11464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:19:56.654776 11453 server_base.cc:1034] running on GCE node
I20260809 02:19:56.655313 11453 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:19:56.655864 11453 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:19:56.657027 11453 hybrid_clock.cc:648] HybridClock initialized: now 1786241996656990 us; error 46 us; skew 500 ppm
Aug 09 02:19:56 dist-test-slave-s11t krb5kdc[10892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241996, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:19:56.662797 11453 init.cc:377] Logged in from keytab as kudu/127.15.75.131@KRBTEST.COM (short username kudu)
I20260809 02:19:56.663642 11453 webserver.cc:460] Webserver started at http://127.15.75.131:44227/ using document root <none> and password file <none>
I20260809 02:19:56.664136 11453 fs_manager.cc:359] Metadata directory not provided
I20260809 02:19:56.664201 11453 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:19:56.664363 11453 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:19:56.666355 11453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/minicluster-data/ts-2/data/instance:
uuid: "2b372e67f03a41f3ab42c035c54affec"
format_stamp: "Formatted at 2026-08-09 02:19:56 on dist-test-slave-s11t"
server_key: "aab29d4b4337a225c7cbe61b5031795d"
server_key_iv: "5ddb4fe576ef2b822c561da4c477d935"
server_key_version: "encryptionkey@0"
I20260809 02:19:56.666858 11453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/minicluster-data/ts-2/wal/instance:
uuid: "2b372e67f03a41f3ab42c035c54affec"
format_stamp: "Formatted at 2026-08-09 02:19:56 on dist-test-slave-s11t"
server_key: "aab29d4b4337a225c7cbe61b5031795d"
server_key_iv: "5ddb4fe576ef2b822c561da4c477d935"
server_key_version: "encryptionkey@0"
I20260809 02:19:56.670399 11453 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260809 02:19:56.672629 11475 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:19:56.673840 11453 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 02:19:56.673974 11453 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/minicluster-data/ts-2/wal
uuid: "2b372e67f03a41f3ab42c035c54affec"
format_stamp: "Formatted at 2026-08-09 02:19:56 on dist-test-slave-s11t"
server_key: "aab29d4b4337a225c7cbe61b5031795d"
server_key_iv: "5ddb4fe576ef2b822c561da4c477d935"
server_key_version: "encryptionkey@0"
I20260809 02:19:56.674082 11453 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:19:56.691485 11453 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:19:56.694243 11453 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:19:56.694443 11453 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:19:56.695017 11453 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:19:56.695829 11453 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:19:56.695911 11453 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:19:56.695986 11453 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:19:56.696022 11453 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:19:56.705310 11453 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.131:44367
I20260809 02:19:56.705343 11648 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.131:44367 every 8 connection(s)
I20260809 02:19:56.706147 11453 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
I20260809 02:19:56.709204 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 11453
I20260809 02:19:56.709338 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/minicluster-data/ts-2/wal/instance
I20260809 02:19:56.709612 15662 external_mini_cluster.cc:1428] Setting key 8098b761691d880fede1cc317a1b5377
Aug 09 02:19:56 dist-test-slave-s11t krb5kdc[10892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241996, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.131@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:19:56.719655 11008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:56.707929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.131:50301 (local address 127.15.75.190:44215)
0809 02:19:56.708064 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:56.708068 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:56.708902 (+ 834us) server_negotiation.cc:407] Connection header received
0809 02:19:56.710268 (+ 1366us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:56.710272 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:56.710328 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:56.710393 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:56.711870 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:56.712419 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:56.713570 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:56.713798 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:56.713918 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:56.716341 (+ 2423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:19:56.716345 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:19:56.716347 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:19:56.716380 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:19:56.717678 (+ 1298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:56.718187 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:56.718191 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:56.718192 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:56.718271 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:56.718617 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:56.718620 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:56.718622 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:56.718800 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:19:56.718916 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:56.719365 (+ 449us) server_negotiation.cc:299] Negotiation successful
0809 02:19:56.719488 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 02:19:56.720346 11654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:56.708287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:44215 (local address 127.15.75.131:50301)
0809 02:19:56.708752 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:56.708791 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 02:19:56.710079 (+ 1288us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:19:56.710624 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:19:56.710632 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:19:56.711110 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:19:56.711726 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:56.711741 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:56.712553 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:56.712556 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:56.713442 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:56.713451 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:56.713943 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:56.713948 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:56.714123 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:56.714646 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:19:56.714661 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:19:56.716186 (+ 1525us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:19:56.717801 (+ 1615us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:19:56.717806 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:19:56.717819 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:19:56.718067 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:19:56.718363 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:19:56.718367 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:19:56.718369 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:19:56.718509 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:19:56.718906 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:19:56.718912 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:19:56.719143 (+ 231us) client_negotiation.cc:769] Sending connection context
0809 02:19:56.719320 (+ 177us) client_negotiation.cc:240] Negotiation successful
0809 02:19:56.719619 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":119,"threads_started":1}
I20260809 02:19:56.721468 11650 heartbeater.cc:347] Connected to a master server at 127.15.75.190:44215
I20260809 02:19:56.721693 11650 heartbeater.cc:464] Registering TS with master...
I20260809 02:19:56.722138 11650 heartbeater.cc:511] Master 127.15.75.190:44215 requested a full tablet report, sending...
I20260809 02:19:56.723239 10954 ts_manager.cc:194] Registered new tserver with Master: 2b372e67f03a41f3ab42c035c54affec (127.15.75.131:44367)
I20260809 02:19:56.723816 10954 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.75.131@KRBTEST.COM'} at 127.15.75.131:50301
I20260809 02:19:56.734973 15662 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 02:19:56.744172 11008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:56.736881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51060 (local address 127.15.75.190:44215)
0809 02:19:56.737018 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:56.737022 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:56.737226 (+ 204us) server_negotiation.cc:407] Connection header received
0809 02:19:56.737315 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:56.737317 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:56.737351 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:56.737418 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:56.738079 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:56.738558 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:56.739372 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:56.739572 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:56.739643 (+ 71us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:56.741165 (+ 1522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:19:56.741171 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:19:56.741176 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:19:56.741208 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:19:56.742434 (+ 1226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:56.742912 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:56.742914 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:56.742915 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:56.742981 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:56.743274 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:56.743276 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:56.743277 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:56.743459 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:19:56.743571 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:56.743794 (+ 223us) server_negotiation.cc:299] Negotiation successful
0809 02:19:56.743997 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 02:19:56.746966 10954 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51060:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:19:56.750106 10954 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:19:56.764336 11668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:56.758083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.131:44367 (local address 127.0.0.1:43978)
0809 02:19:56.758565 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:56.758578 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:19:56.758667 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:19:56.759807 (+ 1140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:19:56.759810 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:19:56.759828 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:19:56.759928 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:56.759937 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:56.761320 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:56.761323 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:56.762884 (+ 1561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:56.762895 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:56.763748 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:56.763752 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:56.763891 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:56.763936 (+ 45us) client_negotiation.cc:769] Sending connection context
0809 02:19:56.764071 (+ 135us) client_negotiation.cc:240] Negotiation successful
0809 02:19:56.764202 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":424,"mutex_wait_us":179,"thread_start_us":95,"threads_started":1}
I20260809 02:19:56.764344 11674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:56.758650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43978 (local address 127.15.75.131:44367)
0809 02:19:56.759391 (+ 741us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:56.759403 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:56.759424 (+ 21us) server_negotiation.cc:407] Connection header received
0809 02:19:56.759494 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:56.759499 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:56.759686 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:56.759831 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:19:56.760033 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:56.761147 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:56.763036 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:56.763607 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:56.763746 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:56.763899 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:56.764104 (+ 205us) server_negotiation.cc:299] Negotiation successful
0809 02:19:56.764199 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":642,"thread_start_us":165,"threads_started":1}
I20260809 02:19:56.764586 11676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:56.758984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36822 (local address 127.15.75.129:43373)
0809 02:19:56.759357 (+ 373us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:56.759374 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:56.759397 (+ 23us) server_negotiation.cc:407] Connection header received
0809 02:19:56.759737 (+ 340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:56.759743 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:56.759919 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:56.760056 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:19:56.760361 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:56.761294 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:56.763014 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:56.763820 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:56.763915 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:56.764046 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:56.764323 (+ 277us) server_negotiation.cc:299] Negotiation successful
0809 02:19:56.764418 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":89,"threads_started":1}
I20260809 02:19:56.764729 11675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:56.758802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.129:43373 (local address 127.0.0.1:36822)
0809 02:19:56.759112 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:56.759126 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:19:56.759571 (+ 445us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:19:56.760176 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:19:56.760178 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:19:56.760191 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:19:56.760248 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:56.760251 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:56.761435 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:56.761439 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:56.762884 (+ 1445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:56.762895 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:56.764114 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:56.764117 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:56.764180 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:56.764197 (+ 17us) client_negotiation.cc:769] Sending connection context
0809 02:19:56.764291 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 02:19:56.764361 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"mutex_wait_us":148,"thread_start_us":102,"threads_started":1}
I20260809 02:19:56.765075 11666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:56.757902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.130:35883 (local address 127.0.0.1:54014)
0809 02:19:56.758254 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:56.758305 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 02:19:56.758420 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:19:56.759332 (+ 912us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:19:56.759338 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:19:56.759363 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:19:56.759512 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:56.759520 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:56.761285 (+ 1765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:56.761288 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:56.762741 (+ 1453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:56.762750 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:56.763853 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:56.763860 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:56.764777 (+ 917us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:56.764802 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 02:19:56.764889 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 02:19:56.764955 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"mutex_wait_us":808,"thread_start_us":82,"threads_started":1}
I20260809 02:19:56.765437 11667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:56.758036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54014 (local address 127.15.75.130:35883)
0809 02:19:56.758786 (+ 750us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:56.758802 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:56.758830 (+ 28us) server_negotiation.cc:407] Connection header received
0809 02:19:56.758912 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:56.758918 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:56.759138 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:56.759672 (+ 534us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:19:56.759865 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:56.761131 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:56.762864 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:56.763703 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:56.763920 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:56.764047 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:56.764950 (+ 903us) server_negotiation.cc:299] Negotiation successful
0809 02:19:56.765302 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":631,"thread_start_us":123,"threads_started":1}
I20260809 02:19:56.767607 11550 tablet_service.cc:1467] Processing CreateTablet for tablet 711aabe5f2b646e69239553ffd38f477 (DEFAULT_TABLE table=test-table [id=5e56576c05164b9aba43540992523e0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:19:56.767611 11137 tablet_service.cc:1467] Processing CreateTablet for tablet 711aabe5f2b646e69239553ffd38f477 (DEFAULT_TABLE table=test-table [id=5e56576c05164b9aba43540992523e0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:19:56.768054 11346 tablet_service.cc:1467] Processing CreateTablet for tablet 711aabe5f2b646e69239553ffd38f477 (DEFAULT_TABLE table=test-table [id=5e56576c05164b9aba43540992523e0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:19:56.768770 11550 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 711aabe5f2b646e69239553ffd38f477. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:19:56.768770 11137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 711aabe5f2b646e69239553ffd38f477. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:19:56.769160 11346 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 711aabe5f2b646e69239553ffd38f477. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:19:56.776362 11677 tablet_bootstrap.cc:492] T 711aabe5f2b646e69239553ffd38f477 P 0838a1e5d0234d0da692fd177343c2eb: Bootstrap starting.
I20260809 02:19:56.776417 11680 tablet_bootstrap.cc:492] T 711aabe5f2b646e69239553ffd38f477 P b3c787537d53481c8846ea80108c1e65: Bootstrap starting.
I20260809 02:19:56.778367 11677 tablet_bootstrap.cc:654] T 711aabe5f2b646e69239553ffd38f477 P 0838a1e5d0234d0da692fd177343c2eb: Neither blocks nor log segments found. Creating new log.
I20260809 02:19:56.778367 11680 tablet_bootstrap.cc:654] T 711aabe5f2b646e69239553ffd38f477 P b3c787537d53481c8846ea80108c1e65: Neither blocks nor log segments found. Creating new log.
I20260809 02:19:56.779525 11677 log.cc:824] T 711aabe5f2b646e69239553ffd38f477 P 0838a1e5d0234d0da692fd177343c2eb: Log is configured to *not* fsync() on all Append() calls
I20260809 02:19:56.779525 11680 log.cc:824] T 711aabe5f2b646e69239553ffd38f477 P b3c787537d53481c8846ea80108c1e65: Log is configured to *not* fsync() on all Append() calls
I20260809 02:19:56.780259 11679 tablet_bootstrap.cc:492] T 711aabe5f2b646e69239553ffd38f477 P 2b372e67f03a41f3ab42c035c54affec: Bootstrap starting.
I20260809 02:19:56.781625 11677 tablet_bootstrap.cc:492] T 711aabe5f2b646e69239553ffd38f477 P 0838a1e5d0234d0da692fd177343c2eb: No bootstrap required, opened a new log
I20260809 02:19:56.781837 11677 ts_tablet_manager.cc:1397] T 711aabe5f2b646e69239553ffd38f477 P 0838a1e5d0234d0da692fd177343c2eb: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 02:19:56.782254 11679 tablet_bootstrap.cc:654] T 711aabe5f2b646e69239553ffd38f477 P 2b372e67f03a41f3ab42c035c54affec: Neither blocks nor log segments found. Creating new log.
I20260809 02:19:56.783285 11679 log.cc:824] T 711aabe5f2b646e69239553ffd38f477 P 2b372e67f03a41f3ab42c035c54affec: Log is configured to *not* fsync() on all Append() calls
I20260809 02:19:56.784840 11680 tablet_bootstrap.cc:492] T 711aabe5f2b646e69239553ffd38f477 P b3c787537d53481c8846ea80108c1e65: No bootstrap required, opened a new log
I20260809 02:19:56.785034 11680 ts_tablet_manager.cc:1397] T 711aabe5f2b646e69239553ffd38f477 P b3c787537d53481c8846ea80108c1e65: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260809 02:19:56.785418 11677 raft_consensus.cc:348] T 711aabe5f2b646e69239553ffd38f477 P 0838a1e5d0234d0da692fd177343c2eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0838a1e5d0234d0da692fd177343c2eb" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 35883 } } peers { permanent_uuid: "2b372e67f03a41f3ab42c035c54affec" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 44367 } } peers { permanent_uuid: "b3c787537d53481c8846ea80108c1e65" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 43373 } }
I20260809 02:19:56.785650 11677 raft_consensus.cc:374] T 711aabe5f2b646e69239553ffd38f477 P 0838a1e5d0234d0da692fd177343c2eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:19:56.785717 11677 raft_consensus.cc:729] T 711aabe5f2b646e69239553ffd38f477 P 0838a1e5d0234d0da692fd177343c2eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0838a1e5d0234d0da692fd177343c2eb, State: Initialized, Role: FOLLOWER
I20260809 02:19:56.786111 11677 consensus_queue.cc:260] T 711aabe5f2b646e69239553ffd38f477 P 0838a1e5d0234d0da692fd177343c2eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0838a1e5d0234d0da692fd177343c2eb" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 35883 } } peers { permanent_uuid: "2b372e67f03a41f3ab42c035c54affec" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 44367 } } peers { permanent_uuid: "b3c787537d53481c8846ea80108c1e65" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 43373 } }
I20260809 02:19:56.786792 11447 heartbeater.cc:503] Master 127.15.75.190:44215 was elected leader, sending a full tablet report...
I20260809 02:19:56.787077 11677 ts_tablet_manager.cc:1428] T 711aabe5f2b646e69239553ffd38f477 P 0838a1e5d0234d0da692fd177343c2eb: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260809 02:19:56.788738 11679 tablet_bootstrap.cc:492] T 711aabe5f2b646e69239553ffd38f477 P 2b372e67f03a41f3ab42c035c54affec: No bootstrap required, opened a new log
I20260809 02:19:56.788940 11679 ts_tablet_manager.cc:1397] T 711aabe5f2b646e69239553ffd38f477 P 2b372e67f03a41f3ab42c035c54affec: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260809 02:19:56.790102 11680 raft_consensus.cc:348] T 711aabe5f2b646e69239553ffd38f477 P b3c787537d53481c8846ea80108c1e65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0838a1e5d0234d0da692fd177343c2eb" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 35883 } } peers { permanent_uuid: "2b372e67f03a41f3ab42c035c54affec" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 44367 } } peers { permanent_uuid: "b3c787537d53481c8846ea80108c1e65" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 43373 } }
I20260809 02:19:56.790357 11680 raft_consensus.cc:374] T 711aabe5f2b646e69239553ffd38f477 P b3c787537d53481c8846ea80108c1e65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:19:56.790442 11680 raft_consensus.cc:729] T 711aabe5f2b646e69239553ffd38f477 P b3c787537d53481c8846ea80108c1e65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3c787537d53481c8846ea80108c1e65, State: Initialized, Role: FOLLOWER
I20260809 02:19:56.790884 11680 consensus_queue.cc:260] T 711aabe5f2b646e69239553ffd38f477 P b3c787537d53481c8846ea80108c1e65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0838a1e5d0234d0da692fd177343c2eb" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 35883 } } peers { permanent_uuid: "2b372e67f03a41f3ab42c035c54affec" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 44367 } } peers { permanent_uuid: "b3c787537d53481c8846ea80108c1e65" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 43373 } }
I20260809 02:19:56.791590 11246 heartbeater.cc:503] Master 127.15.75.190:44215 was elected leader, sending a full tablet report...
I20260809 02:19:56.791891 11680 ts_tablet_manager.cc:1428] T 711aabe5f2b646e69239553ffd38f477 P b3c787537d53481c8846ea80108c1e65: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260809 02:19:56.792277 11679 raft_consensus.cc:348] T 711aabe5f2b646e69239553ffd38f477 P 2b372e67f03a41f3ab42c035c54affec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0838a1e5d0234d0da692fd177343c2eb" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 35883 } } peers { permanent_uuid: "2b372e67f03a41f3ab42c035c54affec" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 44367 } } peers { permanent_uuid: "b3c787537d53481c8846ea80108c1e65" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 43373 } }
I20260809 02:19:56.792493 11679 raft_consensus.cc:374] T 711aabe5f2b646e69239553ffd38f477 P 2b372e67f03a41f3ab42c035c54affec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:19:56.792553 11679 raft_consensus.cc:729] T 711aabe5f2b646e69239553ffd38f477 P 2b372e67f03a41f3ab42c035c54affec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b372e67f03a41f3ab42c035c54affec, State: Initialized, Role: FOLLOWER
I20260809 02:19:56.792953 11679 consensus_queue.cc:260] T 711aabe5f2b646e69239553ffd38f477 P 2b372e67f03a41f3ab42c035c54affec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0838a1e5d0234d0da692fd177343c2eb" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 35883 } } peers { permanent_uuid: "2b372e67f03a41f3ab42c035c54affec" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 44367 } } peers { permanent_uuid: "b3c787537d53481c8846ea80108c1e65" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 43373 } }
I20260809 02:19:56.793592 11650 heartbeater.cc:503] Master 127.15.75.190:44215 was elected leader, sending a full tablet report...
I20260809 02:19:56.793854 11679 ts_tablet_manager.cc:1428] T 711aabe5f2b646e69239553ffd38f477 P 2b372e67f03a41f3ab42c035c54affec: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260809 02:19:56.809062 11692 raft_consensus.cc:482] T 711aabe5f2b646e69239553ffd38f477 P b3c787537d53481c8846ea80108c1e65 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:19:56.809219 11692 raft_consensus.cc:504] T 711aabe5f2b646e69239553ffd38f477 P b3c787537d53481c8846ea80108c1e65 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0838a1e5d0234d0da692fd177343c2eb" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 35883 } } peers { permanent_uuid: "2b372e67f03a41f3ab42c035c54affec" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 44367 } } peers { permanent_uuid: "b3c787537d53481c8846ea80108c1e65" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 43373 } }
I20260809 02:19:56.810189 11692 leader_election.cc:288] T 711aabe5f2b646e69239553ffd38f477 P b3c787537d53481c8846ea80108c1e65 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0838a1e5d0234d0da692fd177343c2eb (127.15.75.130:35883), 2b372e67f03a41f3ab42c035c54affec (127.15.75.131:44367)
I20260809 02:19:56.813957 11674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:56.810527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.129:48561 (local address 127.15.75.131:44367)
0809 02:19:56.810703 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:56.810708 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:56.810755 (+ 47us) server_negotiation.cc:407] Connection header received
0809 02:19:56.810919 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:56.810922 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:56.810970 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:56.811054 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:19:56.811343 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:56.811878 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:56.813005 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:56.813414 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:56.813545 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:56.813597 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:56.813777 (+ 180us) server_negotiation.cc:299] Negotiation successful
0809 02:19:56.813845 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260809 02:19:56.813971 11696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:56.810422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.131:44367 (local address 127.15.75.129:48561)
0809 02:19:56.810682 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:56.810699 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 02:19:56.810808 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:19:56.811068 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:19:56.811071 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:19:56.811090 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:19:56.811216 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:56.811223 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:56.812001 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:56.812006 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:56.812882 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:56.812892 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:56.813541 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:56.813545 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:56.813648 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:56.813668 (+ 20us) client_negotiation.cc:769] Sending connection context
0809 02:19:56.813758 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 02:19:56.813830 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":67,"threads_started":1}
I20260809 02:19:56.814118 11667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:56.810516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.129:34571 (local address 127.15.75.130:35883)
0809 02:19:56.810644 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:56.810649 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:56.810916 (+ 267us) server_negotiation.cc:407] Connection header received
0809 02:19:56.811064 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:56.811069 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:56.811117 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:56.811203 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:19:56.811479 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:56.811971 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:56.813096 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:56.813496 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:56.813637 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:56.813684 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:56.813901 (+ 217us) server_negotiation.cc:299] Negotiation successful
0809 02:19:56.813972 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 02:19:56.814401 11697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:56.810422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.130:35883 (local address 127.15.75.129:34571)
0809 02:19:56.810819 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:56.810831 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:19:56.810954 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:19:56.811215 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:19:56.811218 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:19:56.811238 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:19:56.811368 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:56.811374 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:56.812092 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:56.812095 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:56.812969 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:56.812979 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:56.813633 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:56.813636 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:56.813757 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:56.813791 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 02:19:56.813883 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 02:19:56.813954 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"mutex_wait_us":38,"thread_start_us":107,"threads_started":1}
I20260809 02:19:56.814638 11578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "711aabe5f2b646e69239553ffd38f477" candidate_uuid: "b3c787537d53481c8846ea80108c1e65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b372e67f03a41f3ab42c035c54affec" is_pre_election: true
I20260809 02:19:56.814929 11578 raft_consensus.cc:2454] T 711aabe5f2b646e69239553ffd38f477 P 2b372e67f03a41f3ab42c035c54affec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3c787537d53481c8846ea80108c1e65 in term 0.
I20260809 02:19:56.815059 11378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "711aabe5f2b646e69239553ffd38f477" candidate_uuid: "b3c787537d53481c8846ea80108c1e65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0838a1e5d0234d0da692fd177343c2eb" is_pre_election: true
I20260809 02:19:56.815347 11378 raft_consensus.cc:2454] T 711aabe5f2b646e69239553ffd38f477 P 0838a1e5d0234d0da692fd177343c2eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3c787537d53481c8846ea80108c1e65 in term 0.
I20260809 02:19:56.815390 11077 leader_election.cc:302] T 711aabe5f2b646e69239553ffd38f477 P b3c787537d53481c8846ea80108c1e65 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b372e67f03a41f3ab42c035c54affec, b3c787537d53481c8846ea80108c1e65; no voters:
I20260809 02:19:56.815737 11692 raft_consensus.cc:2781] T 711aabe5f2b646e69239553ffd38f477 P b3c787537d53481c8846ea80108c1e65 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:19:56.815850 11692 raft_consensus.cc:482] T 711aabe5f2b646e69239553ffd38f477 P b3c787537d53481c8846ea80108c1e65 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:19:56.815908 11692 raft_consensus.cc:3037] T 711aabe5f2b646e69239553ffd38f477 P b3c787537d53481c8846ea80108c1e65 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:19:56.817034 11692 raft_consensus.cc:504] T 711aabe5f2b646e69239553ffd38f477 P b3c787537d53481c8846ea80108c1e65 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0838a1e5d0234d0da692fd177343c2eb" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 35883 } } peers { permanent_uuid: "2b372e67f03a41f3ab42c035c54affec" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 44367 } } peers { permanent_uuid: "b3c787537d53481c8846ea80108c1e65" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 43373 } }
I20260809 02:19:56.817376 11692 leader_election.cc:288] T 711aabe5f2b646e69239553ffd38f477 P b3c787537d53481c8846ea80108c1e65 [CANDIDATE]: Term 1 election: Requested vote from peers 0838a1e5d0234d0da692fd177343c2eb (127.15.75.130:35883), 2b372e67f03a41f3ab42c035c54affec (127.15.75.131:44367)
I20260809 02:19:56.817756 11378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "711aabe5f2b646e69239553ffd38f477" candidate_uuid: "b3c787537d53481c8846ea80108c1e65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0838a1e5d0234d0da692fd177343c2eb"
I20260809 02:19:56.817756 11578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "711aabe5f2b646e69239553ffd38f477" candidate_uuid: "b3c787537d53481c8846ea80108c1e65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b372e67f03a41f3ab42c035c54affec"
I20260809 02:19:56.817881 11378 raft_consensus.cc:3037] T 711aabe5f2b646e69239553ffd38f477 P 0838a1e5d0234d0da692fd177343c2eb [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:19:56.817881 11578 raft_consensus.cc:3037] T 711aabe5f2b646e69239553ffd38f477 P 2b372e67f03a41f3ab42c035c54affec [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:19:56.819172 11578 raft_consensus.cc:2454] T 711aabe5f2b646e69239553ffd38f477 P 2b372e67f03a41f3ab42c035c54affec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3c787537d53481c8846ea80108c1e65 in term 1.
I20260809 02:19:56.819172 11378 raft_consensus.cc:2454] T 711aabe5f2b646e69239553ffd38f477 P 0838a1e5d0234d0da692fd177343c2eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3c787537d53481c8846ea80108c1e65 in term 1.
I20260809 02:19:56.819525 11077 leader_election.cc:302] T 711aabe5f2b646e69239553ffd38f477 P b3c787537d53481c8846ea80108c1e65 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b372e67f03a41f3ab42c035c54affec, b3c787537d53481c8846ea80108c1e65; no voters:
I20260809 02:19:56.819677 11692 raft_consensus.cc:2781] T 711aabe5f2b646e69239553ffd38f477 P b3c787537d53481c8846ea80108c1e65 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:19:56.819882 11692 raft_consensus.cc:686] T 711aabe5f2b646e69239553ffd38f477 P b3c787537d53481c8846ea80108c1e65 [term 1 LEADER]: Becoming Leader. State: Replica: b3c787537d53481c8846ea80108c1e65, State: Running, Role: LEADER
I20260809 02:19:56.820176 11692 consensus_queue.cc:237] T 711aabe5f2b646e69239553ffd38f477 P b3c787537d53481c8846ea80108c1e65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0838a1e5d0234d0da692fd177343c2eb" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 35883 } } peers { permanent_uuid: "2b372e67f03a41f3ab42c035c54affec" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 44367 } } peers { permanent_uuid: "b3c787537d53481c8846ea80108c1e65" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 43373 } }
I20260809 02:19:56.823370 10953 catalog_manager.cc:5495] T 711aabe5f2b646e69239553ffd38f477 P b3c787537d53481c8846ea80108c1e65 reported cstate change: term changed from 0 to 1, leader changed from <none> to b3c787537d53481c8846ea80108c1e65 (127.15.75.129). New cstate: current_term: 1 leader_uuid: "b3c787537d53481c8846ea80108c1e65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0838a1e5d0234d0da692fd177343c2eb" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 35883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b372e67f03a41f3ab42c035c54affec" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 44367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3c787537d53481c8846ea80108c1e65" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 43373 } health_report { overall_health: HEALTHY } } }
I20260809 02:19:56.842777 11676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:56.839553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36824 (local address 127.15.75.129:43373)
0809 02:19:56.839665 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:56.839669 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:56.839681 (+ 12us) server_negotiation.cc:407] Connection header received
0809 02:19:56.839827 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:56.839829 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:56.839887 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:56.839974 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:19:56.840175 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:56.840655 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:56.841529 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:56.841728 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:56.841801 (+ 73us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:56.842094 (+ 293us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:19:56.842451 (+ 357us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:19:56.842557 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:56.842638 (+ 81us) server_negotiation.cc:299] Negotiation successful
0809 02:19:56.842669 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 02:19:56.848479 11378 raft_consensus.cc:1261] T 711aabe5f2b646e69239553ffd38f477 P 0838a1e5d0234d0da692fd177343c2eb [term 1 FOLLOWER]: Refusing update from remote peer b3c787537d53481c8846ea80108c1e65: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:19:56.848480 11578 raft_consensus.cc:1261] T 711aabe5f2b646e69239553ffd38f477 P 2b372e67f03a41f3ab42c035c54affec [term 1 FOLLOWER]: Refusing update from remote peer b3c787537d53481c8846ea80108c1e65: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:19:56.849251 11692 consensus_queue.cc:1035] T 711aabe5f2b646e69239553ffd38f477 P b3c787537d53481c8846ea80108c1e65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b372e67f03a41f3ab42c035c54affec" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 44367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:19:56.849457 11698 consensus_queue.cc:1035] T 711aabe5f2b646e69239553ffd38f477 P b3c787537d53481c8846ea80108c1e65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0838a1e5d0234d0da692fd177343c2eb" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 35883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:19:56.857091 11708 mvcc.cc:205] Tried to move back new op lower bound from 7316447219084079104 to 7316447218978435072. Current Snapshot: MvccSnapshot[applied={T|T < 7316447219084079104}]
I20260809 02:19:56.865382 10954 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51060:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:19:56.865521 10954 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51060:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:19:56.867962 10954 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3c64e55e35a34e21ba907f6af834ae02: Sending DeleteTablet for 3 replicas of tablet 711aabe5f2b646e69239553ffd38f477
I20260809 02:19:56.868752 11550 tablet_service.cc:1514] Processing DeleteTablet for tablet 711aabe5f2b646e69239553ffd38f477 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:19:56 UTC) from {username='kudu', principal='kudu/127.15.75.190@KRBTEST.COM'} at 127.0.0.1:43978
I20260809 02:19:56.868746 11346 tablet_service.cc:1514] Processing DeleteTablet for tablet 711aabe5f2b646e69239553ffd38f477 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:19:56 UTC) from {username='kudu', principal='kudu/127.15.75.190@KRBTEST.COM'} at 127.0.0.1:54014
I20260809 02:19:56.869271 11137 tablet_service.cc:1514] Processing DeleteTablet for tablet 711aabe5f2b646e69239553ffd38f477 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:19:56 UTC) from {username='kudu', principal='kudu/127.15.75.190@KRBTEST.COM'} at 127.0.0.1:36822
I20260809 02:19:56.869733 11721 tablet_replica.cc:330] T 711aabe5f2b646e69239553ffd38f477 P b3c787537d53481c8846ea80108c1e65: stopping tablet replica
I20260809 02:19:56.869791 11718 tablet_replica.cc:330] T 711aabe5f2b646e69239553ffd38f477 P 2b372e67f03a41f3ab42c035c54affec: stopping tablet replica
I20260809 02:19:56.870177 11721 raft_consensus.cc:2229] T 711aabe5f2b646e69239553ffd38f477 P b3c787537d53481c8846ea80108c1e65 [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:19:56.870237 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 11047
I20260809 02:19:56.876888 11718 raft_consensus.cc:2229] T 711aabe5f2b646e69239553ffd38f477 P 2b372e67f03a41f3ab42c035c54affec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:19:56.876919 11719 tablet_replica.cc:330] T 711aabe5f2b646e69239553ffd38f477 P 0838a1e5d0234d0da692fd177343c2eb: stopping tablet replica
I20260809 02:19:56.877210 11718 raft_consensus.cc:2258] T 711aabe5f2b646e69239553ffd38f477 P 2b372e67f03a41f3ab42c035c54affec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:19:56.877329 11719 raft_consensus.cc:2229] T 711aabe5f2b646e69239553ffd38f477 P 0838a1e5d0234d0da692fd177343c2eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:19:56.877619 11719 raft_consensus.cc:2258] T 711aabe5f2b646e69239553ffd38f477 P 0838a1e5d0234d0da692fd177343c2eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:19:56.879904 11719 ts_tablet_manager.cc:1905] T 711aabe5f2b646e69239553ffd38f477 P 0838a1e5d0234d0da692fd177343c2eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:19:56.879899 11718 ts_tablet_manager.cc:1905] T 711aabe5f2b646e69239553ffd38f477 P 2b372e67f03a41f3ab42c035c54affec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:19:56.882620 11718 ts_tablet_manager.cc:1918] T 711aabe5f2b646e69239553ffd38f477 P 2b372e67f03a41f3ab42c035c54affec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:19:56.882620 11719 ts_tablet_manager.cc:1918] T 711aabe5f2b646e69239553ffd38f477 P 0838a1e5d0234d0da692fd177343c2eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:19:56.882740 11719 log.cc:1196] T 711aabe5f2b646e69239553ffd38f477 P 0838a1e5d0234d0da692fd177343c2eb: Deleting WAL directory at /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/minicluster-data/ts-1/wal/wals/711aabe5f2b646e69239553ffd38f477
I20260809 02:19:56.882740 11718 log.cc:1196] T 711aabe5f2b646e69239553ffd38f477 P 2b372e67f03a41f3ab42c035c54affec: Deleting WAL directory at /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241741788589-15662-0/minicluster-data/ts-2/wal/wals/711aabe5f2b646e69239553ffd38f477
I20260809 02:19:56.883106 11719 ts_tablet_manager.cc:1939] T 711aabe5f2b646e69239553ffd38f477 P 0838a1e5d0234d0da692fd177343c2eb: Deleting consensus metadata
I20260809 02:19:56.883111 11718 ts_tablet_manager.cc:1939] T 711aabe5f2b646e69239553ffd38f477 P 2b372e67f03a41f3ab42c035c54affec: Deleting consensus metadata
I20260809 02:19:56.884317 10940 catalog_manager.cc:4841] TS 0838a1e5d0234d0da692fd177343c2eb (127.15.75.130:35883): tablet 711aabe5f2b646e69239553ffd38f477 (table test-table [id=5e56576c05164b9aba43540992523e0c]) successfully deleted
I20260809 02:19:56.884347 10938 catalog_manager.cc:4841] TS 2b372e67f03a41f3ab42c035c54affec (127.15.75.131:44367): tablet 711aabe5f2b646e69239553ffd38f477 (table test-table [id=5e56576c05164b9aba43540992523e0c]) successfully deleted
W20260809 02:19:56.885879 10938 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.75.129:43373 (error 108)
I20260809 02:19:56.886677 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 11251
I20260809 02:19:56.886878 11675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:56.886581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.129:43373 (local address 127.0.0.1:36834)
0809 02:19:56.886706 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:56.886783 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.129:43373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260809 02:19:56.888525 10938 catalog_manager.cc:4584] TS b3c787537d53481c8846ea80108c1e65 (127.15.75.129:43373): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 711aabe5f2b646e69239553ffd38f477: Network error: Client connection negotiation failed: client connection to 127.15.75.129:43373: connect: Connection refused (error 111)
I20260809 02:19:56.897871 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 11453
I20260809 02:19:56.906018 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 10911
2026-08-09T02:19:56Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2427 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-09T02:19:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:19:56Z Disabled control of system clock
I20260809 02:19:56.942804 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241741788589-15662-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241741788589-15662-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241741788589-15662-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:38651
--webserver_interface=127.15.75.190
--webserver_port=0
--builtin_ntp_servers=127.15.75.148:46145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.75.190:38651
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 02:19:57.053287 11727 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:19:57.053530 11727 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:19:57.053565 11727 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:19:57.057475 11727 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:19:57.057525 11727 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:19:57.057540 11727 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:19:57.057556 11727 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:19:57.057597 11727 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:19:57.062215 11727 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:46145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241741788589-15662-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241741788589-15662-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.75.190:38651
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:38651
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.75.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11727
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241741788589-15662-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:19:57.063239 11727 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:19:57.064567 11727 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:19:57.073020 11734 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:19:57.073025 11737 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:19:57.073026 11735 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:19:57.073580 11727 server_base.cc:1034] running on GCE node
I20260809 02:19:57.074127 11727 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:19:57.075173 11727 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:19:57.076360 11727 hybrid_clock.cc:648] HybridClock initialized: now 1786241997076333 us; error 41 us; skew 500 ppm
I20260809 02:19:57.078290 11727 webserver.cc:460] Webserver started at http://127.15.75.190:39173/ using document root <none> and password file <none>
I20260809 02:19:57.078857 11727 fs_manager.cc:359] Metadata directory not provided
I20260809 02:19:57.078955 11727 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:19:57.079190 11727 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:19:57.081259 11727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241741788589-15662-0/minicluster-data/master-0/data/instance:
uuid: "8905d472eeb24cc0a8b9c37dd4e8a701"
format_stamp: "Formatted at 2026-08-09 02:19:57 on dist-test-slave-s11t"
server_key: "9fd38d9e35ef85cec355e9792412171e"
server_key_iv: "0d6f08ea3ef2029c80e0620c0c0d5064"
server_key_version: "encryptionkey@0"
I20260809 02:19:57.081815 11727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241741788589-15662-0/minicluster-data/master-0/wal/instance:
uuid: "8905d472eeb24cc0a8b9c37dd4e8a701"
format_stamp: "Formatted at 2026-08-09 02:19:57 on dist-test-slave-s11t"
server_key: "9fd38d9e35ef85cec355e9792412171e"
server_key_iv: "0d6f08ea3ef2029c80e0620c0c0d5064"
server_key_version: "encryptionkey@0"
I20260809 02:19:57.085362 11727 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260809 02:19:57.087625 11746 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:19:57.088871 11727 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 02:19:57.089015 11727 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241741788589-15662-0/minicluster-data/master-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241741788589-15662-0/minicluster-data/master-0/wal
uuid: "8905d472eeb24cc0a8b9c37dd4e8a701"
format_stamp: "Formatted at 2026-08-09 02:19:57 on dist-test-slave-s11t"
server_key: "9fd38d9e35ef85cec355e9792412171e"
server_key_iv: "0d6f08ea3ef2029c80e0620c0c0d5064"
server_key_version: "encryptionkey@0"
I20260809 02:19:57.089120 11727 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241741788589-15662-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241741788589-15662-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241741788589-15662-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:19:57.105986 11727 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:19:57.106587 11727 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:19:57.106769 11727 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:19:57.114259 11727 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.190:38651
I20260809 02:19:57.114269 11816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.190:38651 every 8 connection(s)
I20260809 02:19:57.115159 11727 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
I20260809 02:19:57.118147 11817 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:19:57.118741 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 11727
I20260809 02:19:57.118893 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241741788589-15662-0/minicluster-data/master-0/wal/instance
I20260809 02:19:57.119185 15662 external_mini_cluster.cc:1428] Setting key b5f9a7b41fc5afe4e97fc3530e383d34
I20260809 02:19:57.124797 11817 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8905d472eeb24cc0a8b9c37dd4e8a701: Bootstrap starting.
I20260809 02:19:57.127385 11817 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8905d472eeb24cc0a8b9c37dd4e8a701: Neither blocks nor log segments found. Creating new log.
I20260809 02:19:57.128279 11817 log.cc:824] T 00000000000000000000000000000000 P 8905d472eeb24cc0a8b9c37dd4e8a701: Log is configured to *not* fsync() on all Append() calls
I20260809 02:19:57.129567 11823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:57.120591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60084 (local address 127.15.75.190:38651)
0809 02:19:57.121296 (+ 705us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:57.121317 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:57.121362 (+ 45us) server_negotiation.cc:407] Connection header received
0809 02:19:57.122214 (+ 852us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:57.122238 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:57.122726 (+ 488us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:57.123092 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:57.123524 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:57.125010 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:57.125970 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:57.126661 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:57.126832 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:57.127642 (+ 810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:19:57.127650 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:19:57.127668 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:19:57.128155 (+ 487us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:19:57.128219 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:19:57.128245 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:19:57.128347 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:57.128485 (+ 138us) server_negotiation.cc:299] Negotiation successful
0809 02:19:57.128796 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":111,"threads_started":1}
I20260809 02:19:57.130244 11817 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8905d472eeb24cc0a8b9c37dd4e8a701: No bootstrap required, opened a new log
I20260809 02:19:57.133742 11817 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8905d472eeb24cc0a8b9c37dd4e8a701 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8905d472eeb24cc0a8b9c37dd4e8a701" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 38651 } }
I20260809 02:19:57.134019 11817 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8905d472eeb24cc0a8b9c37dd4e8a701 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:19:57.134102 11817 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8905d472eeb24cc0a8b9c37dd4e8a701 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8905d472eeb24cc0a8b9c37dd4e8a701, State: Initialized, Role: FOLLOWER
I20260809 02:19:57.134512 11817 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8905d472eeb24cc0a8b9c37dd4e8a701 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8905d472eeb24cc0a8b9c37dd4e8a701" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 38651 } }
I20260809 02:19:57.134658 11817 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8905d472eeb24cc0a8b9c37dd4e8a701 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:19:57.134745 11817 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8905d472eeb24cc0a8b9c37dd4e8a701 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:19:57.134845 11817 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8905d472eeb24cc0a8b9c37dd4e8a701 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:19:57.135852 11817 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8905d472eeb24cc0a8b9c37dd4e8a701 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8905d472eeb24cc0a8b9c37dd4e8a701" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 38651 } }
I20260809 02:19:57.136129 11817 leader_election.cc:302] T 00000000000000000000000000000000 P 8905d472eeb24cc0a8b9c37dd4e8a701 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8905d472eeb24cc0a8b9c37dd4e8a701; no voters:
I20260809 02:19:57.136384 11817 leader_election.cc:288] T 00000000000000000000000000000000 P 8905d472eeb24cc0a8b9c37dd4e8a701 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:19:57.136477 11827 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8905d472eeb24cc0a8b9c37dd4e8a701 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:19:57.136732 11827 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8905d472eeb24cc0a8b9c37dd4e8a701 [term 1 LEADER]: Becoming Leader. State: Replica: 8905d472eeb24cc0a8b9c37dd4e8a701, State: Running, Role: LEADER
I20260809 02:19:57.137053 11827 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8905d472eeb24cc0a8b9c37dd4e8a701 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8905d472eeb24cc0a8b9c37dd4e8a701" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 38651 } }
I20260809 02:19:57.137574 11817 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8905d472eeb24cc0a8b9c37dd4e8a701 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:19:57.138666 11828 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8905d472eeb24cc0a8b9c37dd4e8a701 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8905d472eeb24cc0a8b9c37dd4e8a701" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8905d472eeb24cc0a8b9c37dd4e8a701" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 38651 } } }
I20260809 02:19:57.138792 11828 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8905d472eeb24cc0a8b9c37dd4e8a701 [sys.catalog]: This master's current role is: LEADER
I20260809 02:19:57.139081 11830 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8905d472eeb24cc0a8b9c37dd4e8a701 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8905d472eeb24cc0a8b9c37dd4e8a701. Latest consensus state: current_term: 1 leader_uuid: "8905d472eeb24cc0a8b9c37dd4e8a701" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8905d472eeb24cc0a8b9c37dd4e8a701" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 38651 } } }
I20260809 02:19:57.139184 11838 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:19:57.139184 11830 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8905d472eeb24cc0a8b9c37dd4e8a701 [sys.catalog]: This master's current role is: LEADER
I20260809 02:19:57.141824 11838 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:19:57.146548 11838 catalog_manager.cc:1347] Generated new cluster ID: 98c6df058ead496e96bd4f45f306965c
I20260809 02:19:57.146610 11838 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:19:57.156821 11838 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:19:57.157932 11838 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:19:57.176837 11838 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8905d472eeb24cc0a8b9c37dd4e8a701: Generated new TSK 0
I20260809 02:19:57.177573 11838 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:19:57.182439 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241741788589-15662-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241741788589-15662-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241741788589-15662-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.129:0
--local_ip_for_outbound_sockets=127.15.75.129
--webserver_interface=127.15.75.129
--webserver_port=0
--tserver_master_addrs=127.15.75.190:38651
--builtin_ntp_servers=127.15.75.148:46145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 02:19:57.292726 11853 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:19:57.293025 11853 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:19:57.293100 11853 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:19:57.297068 11853 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:19:57.297132 11853 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:19:57.297214 11853 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.129
I20260809 02:19:57.301841 11853 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:46145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241741788589-15662-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241741788589-15662-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.75.129
--webserver_port=0
--tserver_master_addrs=127.15.75.190:38651
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11853
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.129
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241741788589-15662-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:19:57.302817 11853 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:19:57.304129 11853 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:19:57.313740 11863 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:19:57.313748 11860 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:19:57.313773 11859 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:19:57.314081 11853 server_base.cc:1034] running on GCE node
I20260809 02:19:57.314517 11853 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:19:57.315155 11853 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:19:57.316354 11853 hybrid_clock.cc:648] HybridClock initialized: now 1786241997316324 us; error 40 us; skew 500 ppm
I20260809 02:19:57.318312 11853 webserver.cc:460] Webserver started at http://127.15.75.129:43685/ using document root <none> and password file <none>
I20260809 02:19:57.318945 11853 fs_manager.cc:359] Metadata directory not provided
I20260809 02:19:57.319039 11853 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:19:57.319272 11853 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:19:57.321342 11853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241741788589-15662-0/minicluster-data/ts-0/data/instance:
uuid: "25c56c5acf30469e983da9033bebfaa7"
format_stamp: "Formatted at 2026-08-09 02:19:57 on dist-test-slave-s11t"
server_key: "e93eaa9695048037c138bf7faf4e4c90"
server_key_iv: "63a385fa6deeb5ace634859ab32814ea"
server_key_version: "encryptionkey@0"
I20260809 02:19:57.321930 11853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241741788589-15662-0/minicluster-data/ts-0/wal/instance:
uuid: "25c56c5acf30469e983da9033bebfaa7"
format_stamp: "Formatted at 2026-08-09 02:19:57 on dist-test-slave-s11t"
server_key: "e93eaa9695048037c138bf7faf4e4c90"
server_key_iv: "63a385fa6deeb5ace634859ab32814ea"
server_key_version: "encryptionkey@0"
I20260809 02:19:57.325498 11853 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 02:19:57.327978 11875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:19:57.329254 11853 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 02:19:57.329384 11853 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241741788589-15662-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241741788589-15662-0/minicluster-data/ts-0/wal
uuid: "25c56c5acf30469e983da9033bebfaa7"
format_stamp: "Formatted at 2026-08-09 02:19:57 on dist-test-slave-s11t"
server_key: "e93eaa9695048037c138bf7faf4e4c90"
server_key_iv: "63a385fa6deeb5ace634859ab32814ea"
server_key_version: "encryptionkey@0"
I20260809 02:19:57.329492 11853 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241741788589-15662-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241741788589-15662-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241741788589-15662-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:19:57.346447 11853 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:19:57.347077 11853 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:19:57.347270 11853 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:19:57.347805 11853 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:19:57.348671 11853 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:19:57.348824 11853 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:19:57.348901 11853 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:19:57.348944 11853 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:19:57.357851 11853 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.129:34533
I20260809 02:19:57.357885 12036 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.129:34533 every 8 connection(s)
I20260809 02:19:57.358690 11853 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
I20260809 02:19:57.359453 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 11853
I20260809 02:19:57.359601 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241741788589-15662-0/minicluster-data/ts-0/wal/instance
I20260809 02:19:57.359903 15662 external_mini_cluster.cc:1428] Setting key c31480bcbf2eaa1deb129555856466ba
I20260809 02:19:57.362090 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241741788589-15662-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241741788589-15662-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241741788589-15662-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.130:0
--local_ip_for_outbound_sockets=127.15.75.130
--webserver_interface=127.15.75.130
--webserver_port=0
--tserver_master_addrs=127.15.75.190:38651
--builtin_ntp_servers=127.15.75.148:46145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 02:19:57.368037 11823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:57.360580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.129:40599 (local address 127.15.75.190:38651)
0809 02:19:57.360733 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:57.360738 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:57.361910 (+ 1172us) server_negotiation.cc:407] Connection header received
0809 02:19:57.363382 (+ 1472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:57.363393 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:57.363454 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:57.363531 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:57.364351 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:57.364985 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:57.366111 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:57.366351 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:57.366464 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:57.367537 (+ 1073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:19:57.367541 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:19:57.367544 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:19:57.367556 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:19:57.367579 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:19:57.367594 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:19:57.367672 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:57.367834 (+ 162us) server_negotiation.cc:299] Negotiation successful
0809 02:19:57.367900 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 02:19:57.368875 12039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:57.361039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:38651 (local address 127.15.75.129:40599)
0809 02:19:57.361767 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:57.361805 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 02:19:57.363178 (+ 1373us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:19:57.363688 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:19:57.363694 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:19:57.363771 (+ 77us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:19:57.364208 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:57.364220 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:57.365109 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:57.365115 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:57.365982 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:57.365991 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:57.366503 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:57.366507 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:57.366668 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:57.367284 (+ 616us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:19:57.367285 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:19:57.367323 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:19:57.367381 (+ 58us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:19:57.367675 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:19:57.367687 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:19:57.367696 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 02:19:57.367828 (+ 132us) client_negotiation.cc:240] Negotiation successful
0809 02:19:57.368097 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":161,"threads_started":1}
I20260809 02:19:57.370090 12037 heartbeater.cc:347] Connected to a master server at 127.15.75.190:38651
I20260809 02:19:57.370352 12037 heartbeater.cc:464] Registering TS with master...
I20260809 02:19:57.371001 12037 heartbeater.cc:511] Master 127.15.75.190:38651 requested a full tablet report, sending...
I20260809 02:19:57.372803 11769 ts_manager.cc:194] Registered new tserver with Master: 25c56c5acf30469e983da9033bebfaa7 (127.15.75.129:34533)
I20260809 02:19:57.374521 11769 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.75.129:40599
W20260809 02:19:57.474537 12041 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:19:57.474766 12041 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:19:57.474802 12041 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:19:57.478663 12041 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:19:57.478708 12041 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:19:57.478773 12041 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.130
I20260809 02:19:57.483538 12041 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:46145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241741788589-15662-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241741788589-15662-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.75.130
--webserver_port=0
--tserver_master_addrs=127.15.75.190:38651
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12041
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.130
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241741788589-15662-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:19:57.484485 12041 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:19:57.485806 12041 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:19:57.495157 12050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:19:57.495302 12041 server_base.cc:1034] running on GCE node
W20260809 02:19:57.495220 12048 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:19:57.495208 12047 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:19:57.495817 12041 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:19:57.496397 12041 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:19:57.497565 12041 hybrid_clock.cc:648] HybridClock initialized: now 1786241997497518 us; error 55 us; skew 500 ppm
I20260809 02:19:57.499266 12041 webserver.cc:460] Webserver started at http://127.15.75.130:39053/ using document root <none> and password file <none>
I20260809 02:19:57.499810 12041 fs_manager.cc:359] Metadata directory not provided
I20260809 02:19:57.499871 12041 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:19:57.500123 12041 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:19:57.502110 12041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241741788589-15662-0/minicluster-data/ts-1/data/instance:
uuid: "0d47f58aa3ee4663908c4bbe6d903b04"
format_stamp: "Formatted at 2026-08-09 02:19:57 on dist-test-slave-s11t"
server_key: "abc175927a0f1029fff1865aeef822e1"
server_key_iv: "42211a6c7aaa02ee1213c5fab21dad13"
server_key_version: "encryptionkey@0"
I20260809 02:19:57.502669 12041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241741788589-15662-0/minicluster-data/ts-1/wal/instance:
uuid: "0d47f58aa3ee4663908c4bbe6d903b04"
format_stamp: "Formatted at 2026-08-09 02:19:57 on dist-test-slave-s11t"
server_key: "abc175927a0f1029fff1865aeef822e1"
server_key_iv: "42211a6c7aaa02ee1213c5fab21dad13"
server_key_version: "encryptionkey@0"
I20260809 02:19:57.506251 12041 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 02:19:57.508625 12056 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:19:57.509829 12041 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 02:19:57.509948 12041 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241741788589-15662-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241741788589-15662-0/minicluster-data/ts-1/wal
uuid: "0d47f58aa3ee4663908c4bbe6d903b04"
format_stamp: "Formatted at 2026-08-09 02:19:57 on dist-test-slave-s11t"
server_key: "abc175927a0f1029fff1865aeef822e1"
server_key_iv: "42211a6c7aaa02ee1213c5fab21dad13"
server_key_version: "encryptionkey@0"
I20260809 02:19:57.510054 12041 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241741788589-15662-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241741788589-15662-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241741788589-15662-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:19:57.529358 12041 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:19:57.529956 12041 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:19:57.530141 12041 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:19:57.530697 12041 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:19:57.531582 12041 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:19:57.531628 12041 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:19:57.531704 12041 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:19:57.531761 12041 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:19:57.541142 12041 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.130:41679
I20260809 02:19:57.541177 12216 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.130:41679 every 8 connection(s)
I20260809 02:19:57.541988 12041 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
I20260809 02:19:57.548954 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 12041
I20260809 02:19:57.549108 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241741788589-15662-0/minicluster-data/ts-1/wal/instance
I20260809 02:19:57.549405 15662 external_mini_cluster.cc:1428] Setting key 81eb5fb850253a03d5dbac70c4d208cb
I20260809 02:19:57.551487 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241741788589-15662-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241741788589-15662-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241741788589-15662-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.131:0
--local_ip_for_outbound_sockets=127.15.75.131
--webserver_interface=127.15.75.131
--webserver_port=0
--tserver_master_addrs=127.15.75.190:38651
--builtin_ntp_servers=127.15.75.148:46145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 02:19:57.551939 12219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:57.544134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:38651 (local address 127.15.75.130:47509)
0809 02:19:57.544821 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:57.544859 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 02:19:57.546121 (+ 1262us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:19:57.546666 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:19:57.546672 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:19:57.546747 (+ 75us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:19:57.547180 (+ 433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:57.547192 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:57.547992 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:57.547997 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:57.548880 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:57.548898 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:57.549489 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:57.549493 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:57.549659 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:57.550366 (+ 707us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:19:57.550368 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:19:57.550425 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:19:57.550497 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:19:57.550821 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:19:57.550834 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:19:57.550842 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 02:19:57.550982 (+ 140us) client_negotiation.cc:240] Negotiation successful
0809 02:19:57.551204 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":122,"threads_started":1}
I20260809 02:19:57.552050 11823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:57.543746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.130:47509 (local address 127.15.75.190:38651)
0809 02:19:57.543891 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:57.543895 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:57.544970 (+ 1075us) server_negotiation.cc:407] Connection header received
0809 02:19:57.546326 (+ 1356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:57.546329 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:57.546398 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:57.546501 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:57.547325 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:57.547861 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:57.549016 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:57.549334 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:57.549453 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:57.550650 (+ 1197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:19:57.550653 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:19:57.550655 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:19:57.550689 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:19:57.550715 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:19:57.550726 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:19:57.550803 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:57.551785 (+ 982us) server_negotiation.cc:299] Negotiation successful
0809 02:19:57.551856 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 02:19:57.553037 12217 heartbeater.cc:347] Connected to a master server at 127.15.75.190:38651
I20260809 02:19:57.553275 12217 heartbeater.cc:464] Registering TS with master...
I20260809 02:19:57.553805 12217 heartbeater.cc:511] Master 127.15.75.190:38651 requested a full tablet report, sending...
I20260809 02:19:57.554860 11769 ts_manager.cc:194] Registered new tserver with Master: 0d47f58aa3ee4663908c4bbe6d903b04 (127.15.75.130:41679)
I20260809 02:19:57.555449 11769 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.75.130:47509
W20260809 02:19:57.662704 12222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:19:57.662978 12222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:19:57.663049 12222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:19:57.666975 12222 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:19:57.667049 12222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:19:57.667133 12222 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.131
I20260809 02:19:57.671777 12222 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:46145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241741788589-15662-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241741788589-15662-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.75.131
--webserver_port=0
--tserver_master_addrs=127.15.75.190:38651
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12222
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.131
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241741788589-15662-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:19:57.672765 12222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:19:57.674074 12222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 02:19:57.683684 12222 server_base.cc:1034] running on GCE node
W20260809 02:19:57.683635 12229 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:19:57.683665 12228 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:19:57.683636 12231 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:19:57.684316 12222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:19:57.684959 12222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:19:57.686122 12222 hybrid_clock.cc:648] HybridClock initialized: now 1786241997686087 us; error 45 us; skew 500 ppm
I20260809 02:19:57.687891 12222 webserver.cc:460] Webserver started at http://127.15.75.131:33597/ using document root <none> and password file <none>
I20260809 02:19:57.688441 12222 fs_manager.cc:359] Metadata directory not provided
I20260809 02:19:57.688501 12222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:19:57.688771 12222 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:19:57.690742 12222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241741788589-15662-0/minicluster-data/ts-2/data/instance:
uuid: "408ec93995d343c9b2efdd1709e418f2"
format_stamp: "Formatted at 2026-08-09 02:19:57 on dist-test-slave-s11t"
server_key: "06d5f8ab88c833ac59086326f7c51ebc"
server_key_iv: "351d6867986a2d24eb956b9f658f58e5"
server_key_version: "encryptionkey@0"
I20260809 02:19:57.691309 12222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241741788589-15662-0/minicluster-data/ts-2/wal/instance:
uuid: "408ec93995d343c9b2efdd1709e418f2"
format_stamp: "Formatted at 2026-08-09 02:19:57 on dist-test-slave-s11t"
server_key: "06d5f8ab88c833ac59086326f7c51ebc"
server_key_iv: "351d6867986a2d24eb956b9f658f58e5"
server_key_version: "encryptionkey@0"
I20260809 02:19:57.695214 12222 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260809 02:19:57.697501 12244 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:19:57.698653 12222 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 02:19:57.698773 12222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241741788589-15662-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241741788589-15662-0/minicluster-data/ts-2/wal
uuid: "408ec93995d343c9b2efdd1709e418f2"
format_stamp: "Formatted at 2026-08-09 02:19:57 on dist-test-slave-s11t"
server_key: "06d5f8ab88c833ac59086326f7c51ebc"
server_key_iv: "351d6867986a2d24eb956b9f658f58e5"
server_key_version: "encryptionkey@0"
I20260809 02:19:57.698907 12222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241741788589-15662-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241741788589-15662-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241741788589-15662-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:19:57.719049 12222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:19:57.719650 12222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:19:57.719844 12222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:19:57.720392 12222 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:19:57.721346 12222 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:19:57.721396 12222 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:19:57.721433 12222 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:19:57.721487 12222 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:19:57.730515 12222 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.131:37451
I20260809 02:19:57.730554 12413 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.131:37451 every 8 connection(s)
I20260809 02:19:57.731341 12222 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
I20260809 02:19:57.739427 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 12222
I20260809 02:19:57.739562 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241741788589-15662-0/minicluster-data/ts-2/wal/instance
I20260809 02:19:57.739917 15662 external_mini_cluster.cc:1428] Setting key 2cffd281a2e219867322490cddef3496
I20260809 02:19:57.740967 11823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:57.733114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.131:60749 (local address 127.15.75.190:38651)
0809 02:19:57.733251 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:57.733262 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:57.734367 (+ 1105us) server_negotiation.cc:407] Connection header received
0809 02:19:57.735773 (+ 1406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:57.735777 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:57.735837 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:57.735945 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:57.736818 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:57.737366 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:57.738614 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:57.738858 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:57.738973 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:57.740353 (+ 1380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:19:57.740360 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:19:57.740366 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:19:57.740384 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:19:57.740411 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:19:57.740423 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:19:57.740499 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:57.740753 (+ 254us) server_negotiation.cc:299] Negotiation successful
0809 02:19:57.740813 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 02:19:57.741840 12416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:57.733562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:38651 (local address 127.15.75.131:60749)
0809 02:19:57.734226 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:57.734264 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 02:19:57.735531 (+ 1267us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:19:57.736114 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:19:57.736120 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:19:57.736198 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:19:57.736641 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:57.736652 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:57.737496 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:57.737500 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:57.738489 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:57.738499 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:57.739009 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:57.739013 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:57.739268 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:57.740101 (+ 833us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:19:57.740103 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:19:57.740141 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:19:57.740201 (+ 60us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:19:57.740528 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:19:57.740538 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:19:57.740545 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 02:19:57.740719 (+ 174us) client_negotiation.cc:240] Negotiation successful
0809 02:19:57.740996 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":103,"threads_started":1}
I20260809 02:19:57.742878 12414 heartbeater.cc:347] Connected to a master server at 127.15.75.190:38651
I20260809 02:19:57.743070 12414 heartbeater.cc:464] Registering TS with master...
I20260809 02:19:57.743516 12414 heartbeater.cc:511] Master 127.15.75.190:38651 requested a full tablet report, sending...
I20260809 02:19:57.744450 11769 ts_manager.cc:194] Registered new tserver with Master: 408ec93995d343c9b2efdd1709e418f2 (127.15.75.131:37451)
I20260809 02:19:57.745043 11769 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.75.131:60749
I20260809 02:19:57.753669 15662 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 02:19:57.760174 11823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:57.755564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60092 (local address 127.15.75.190:38651)
0809 02:19:57.755679 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:57.755683 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:57.755900 (+ 217us) server_negotiation.cc:407] Connection header received
0809 02:19:57.755991 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:57.755993 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:57.756028 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:57.756093 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:57.756371 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:57.757021 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:57.757915 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:57.758158 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:57.758301 (+ 143us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:57.759598 (+ 1297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:19:57.759602 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:19:57.759605 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:19:57.759619 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:19:57.759648 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:19:57.759657 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:19:57.759844 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:57.759982 (+ 138us) server_negotiation.cc:299] Negotiation successful
0809 02:19:57.760020 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 02:19:57.762907 11769 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60092:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:19:57.765880 11769 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:19:57.779994 12438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:57.775039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38330 (local address 127.15.75.130:41679)
0809 02:19:57.775470 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:57.775481 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:57.775828 (+ 347us) server_negotiation.cc:407] Connection header received
0809 02:19:57.776016 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:57.776020 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:57.776159 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:57.776306 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:19:57.776483 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:57.777446 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:57.778785 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:57.779299 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:57.779424 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:57.779614 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:57.779711 (+ 97us) server_negotiation.cc:299] Negotiation successful
0809 02:19:57.779811 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":140,"threads_started":1}
I20260809 02:19:57.779989 12430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:57.774009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.129:34533 (local address 127.0.0.1:48348)
0809 02:19:57.774610 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:57.774625 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:19:57.774718 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:19:57.775461 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:19:57.775464 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:19:57.775483 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:19:57.775784 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:57.775807 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:57.777181 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:57.777185 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:57.778460 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:57.778471 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:57.779478 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:57.779482 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:57.779616 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:57.779638 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 02:19:57.779703 (+ 65us) client_negotiation.cc:240] Negotiation successful
0809 02:19:57.779833 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"mutex_wait_us":277,"thread_start_us":108,"threads_started":1}
I20260809 02:19:57.780198 12433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:57.774187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60026 (local address 127.15.75.131:37451)
0809 02:19:57.774872 (+ 685us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:57.774885 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:57.774909 (+ 24us) server_negotiation.cc:407] Connection header received
0809 02:19:57.774986 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:57.774991 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:57.775194 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:57.775615 (+ 421us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:19:57.775752 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:57.776947 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:57.778550 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:57.779327 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:57.779604 (+ 277us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:57.779791 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:57.779949 (+ 158us) server_negotiation.cc:299] Negotiation successful
0809 02:19:57.780040 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":575,"thread_start_us":159,"threads_started":1}
I20260809 02:19:57.780407 12432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:57.773918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.131:37451 (local address 127.0.0.1:60026)
0809 02:19:57.774404 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:57.774451 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 02:19:57.774577 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:19:57.775394 (+ 817us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:19:57.775402 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:19:57.775426 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:19:57.775649 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:57.775659 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:57.777063 (+ 1404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:57.777067 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:57.778422 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:57.778437 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:57.779478 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:57.779482 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:57.779805 (+ 323us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:57.779835 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 02:19:57.779916 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 02:19:57.779988 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"mutex_wait_us":261,"thread_start_us":82,"threads_started":1}
I20260809 02:19:57.780570 12437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:57.774544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48348 (local address 127.15.75.129:34533)
0809 02:19:57.774967 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:57.774980 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:57.775004 (+ 24us) server_negotiation.cc:407] Connection header received
0809 02:19:57.775086 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:57.775093 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:57.775283 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:57.775641 (+ 358us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:19:57.775928 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:57.777064 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:57.778596 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:57.779327 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:57.780045 (+ 718us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:57.780224 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:57.780285 (+ 61us) server_negotiation.cc:299] Negotiation successful
0809 02:19:57.780390 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":152,"threads_started":1}
I20260809 02:19:57.780798 12436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:57.774598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.130:41679 (local address 127.0.0.1:38330)
0809 02:19:57.775725 (+ 1127us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:57.775738 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:19:57.775860 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:19:57.776289 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:19:57.776292 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:19:57.776310 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:19:57.776389 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:57.776395 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:57.777568 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:57.777571 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:57.778672 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:57.778680 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:57.779424 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:57.779428 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:57.779532 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:57.779568 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 02:19:57.779680 (+ 112us) client_negotiation.cc:240] Negotiation successful
0809 02:19:57.780637 (+ 957us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1068,"mutex_wait_us":30,"thread_start_us":100,"threads_started":1}
I20260809 02:19:57.783082 12312 tablet_service.cc:1467] Processing CreateTablet for tablet de04ab0897e943b4aba9dc9985677f24 (DEFAULT_TABLE table=test-table [id=b52f77a893624d4d8df5df56a61a6d4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:19:57.783075 11939 tablet_service.cc:1467] Processing CreateTablet for tablet de04ab0897e943b4aba9dc9985677f24 (DEFAULT_TABLE table=test-table [id=b52f77a893624d4d8df5df56a61a6d4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:19:57.784003 12312 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de04ab0897e943b4aba9dc9985677f24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:19:57.784031 11939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de04ab0897e943b4aba9dc9985677f24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:19:57.787356 12132 tablet_service.cc:1467] Processing CreateTablet for tablet de04ab0897e943b4aba9dc9985677f24 (DEFAULT_TABLE table=test-table [id=b52f77a893624d4d8df5df56a61a6d4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:19:57.788328 12132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de04ab0897e943b4aba9dc9985677f24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:19:57.790766 12440 tablet_bootstrap.cc:492] T de04ab0897e943b4aba9dc9985677f24 P 25c56c5acf30469e983da9033bebfaa7: Bootstrap starting.
I20260809 02:19:57.791275 12441 tablet_bootstrap.cc:492] T de04ab0897e943b4aba9dc9985677f24 P 408ec93995d343c9b2efdd1709e418f2: Bootstrap starting.
I20260809 02:19:57.793408 12440 tablet_bootstrap.cc:654] T de04ab0897e943b4aba9dc9985677f24 P 25c56c5acf30469e983da9033bebfaa7: Neither blocks nor log segments found. Creating new log.
I20260809 02:19:57.793397 12441 tablet_bootstrap.cc:654] T de04ab0897e943b4aba9dc9985677f24 P 408ec93995d343c9b2efdd1709e418f2: Neither blocks nor log segments found. Creating new log.
I20260809 02:19:57.794284 12441 log.cc:824] T de04ab0897e943b4aba9dc9985677f24 P 408ec93995d343c9b2efdd1709e418f2: Log is configured to *not* fsync() on all Append() calls
I20260809 02:19:57.794284 12440 log.cc:824] T de04ab0897e943b4aba9dc9985677f24 P 25c56c5acf30469e983da9033bebfaa7: Log is configured to *not* fsync() on all Append() calls
I20260809 02:19:57.794991 12444 tablet_bootstrap.cc:492] T de04ab0897e943b4aba9dc9985677f24 P 0d47f58aa3ee4663908c4bbe6d903b04: Bootstrap starting.
I20260809 02:19:57.796321 12441 tablet_bootstrap.cc:492] T de04ab0897e943b4aba9dc9985677f24 P 408ec93995d343c9b2efdd1709e418f2: No bootstrap required, opened a new log
I20260809 02:19:57.796491 12440 tablet_bootstrap.cc:492] T de04ab0897e943b4aba9dc9985677f24 P 25c56c5acf30469e983da9033bebfaa7: No bootstrap required, opened a new log
I20260809 02:19:57.796511 12441 ts_tablet_manager.cc:1397] T de04ab0897e943b4aba9dc9985677f24 P 408ec93995d343c9b2efdd1709e418f2: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 02:19:57.796702 12440 ts_tablet_manager.cc:1397] T de04ab0897e943b4aba9dc9985677f24 P 25c56c5acf30469e983da9033bebfaa7: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260809 02:19:57.797792 12444 tablet_bootstrap.cc:654] T de04ab0897e943b4aba9dc9985677f24 P 0d47f58aa3ee4663908c4bbe6d903b04: Neither blocks nor log segments found. Creating new log.
I20260809 02:19:57.798466 12444 log.cc:824] T de04ab0897e943b4aba9dc9985677f24 P 0d47f58aa3ee4663908c4bbe6d903b04: Log is configured to *not* fsync() on all Append() calls
I20260809 02:19:57.800125 12444 tablet_bootstrap.cc:492] T de04ab0897e943b4aba9dc9985677f24 P 0d47f58aa3ee4663908c4bbe6d903b04: No bootstrap required, opened a new log
I20260809 02:19:57.800323 12444 ts_tablet_manager.cc:1397] T de04ab0897e943b4aba9dc9985677f24 P 0d47f58aa3ee4663908c4bbe6d903b04: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260809 02:19:57.800829 12441 raft_consensus.cc:348] T de04ab0897e943b4aba9dc9985677f24 P 408ec93995d343c9b2efdd1709e418f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408ec93995d343c9b2efdd1709e418f2" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 37451 } } peers { permanent_uuid: "25c56c5acf30469e983da9033bebfaa7" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 34533 } } peers { permanent_uuid: "0d47f58aa3ee4663908c4bbe6d903b04" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 41679 } }
I20260809 02:19:57.800829 12440 raft_consensus.cc:348] T de04ab0897e943b4aba9dc9985677f24 P 25c56c5acf30469e983da9033bebfaa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408ec93995d343c9b2efdd1709e418f2" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 37451 } } peers { permanent_uuid: "25c56c5acf30469e983da9033bebfaa7" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 34533 } } peers { permanent_uuid: "0d47f58aa3ee4663908c4bbe6d903b04" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 41679 } }
I20260809 02:19:57.801084 12440 raft_consensus.cc:374] T de04ab0897e943b4aba9dc9985677f24 P 25c56c5acf30469e983da9033bebfaa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:19:57.801084 12441 raft_consensus.cc:374] T de04ab0897e943b4aba9dc9985677f24 P 408ec93995d343c9b2efdd1709e418f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:19:57.801143 12440 raft_consensus.cc:729] T de04ab0897e943b4aba9dc9985677f24 P 25c56c5acf30469e983da9033bebfaa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25c56c5acf30469e983da9033bebfaa7, State: Initialized, Role: FOLLOWER
I20260809 02:19:57.801152 12441 raft_consensus.cc:729] T de04ab0897e943b4aba9dc9985677f24 P 408ec93995d343c9b2efdd1709e418f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 408ec93995d343c9b2efdd1709e418f2, State: Initialized, Role: FOLLOWER
I20260809 02:19:57.801585 12441 consensus_queue.cc:260] T de04ab0897e943b4aba9dc9985677f24 P 408ec93995d343c9b2efdd1709e418f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408ec93995d343c9b2efdd1709e418f2" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 37451 } } peers { permanent_uuid: "25c56c5acf30469e983da9033bebfaa7" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 34533 } } peers { permanent_uuid: "0d47f58aa3ee4663908c4bbe6d903b04" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 41679 } }
I20260809 02:19:57.801607 12440 consensus_queue.cc:260] T de04ab0897e943b4aba9dc9985677f24 P 25c56c5acf30469e983da9033bebfaa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408ec93995d343c9b2efdd1709e418f2" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 37451 } } peers { permanent_uuid: "25c56c5acf30469e983da9033bebfaa7" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 34533 } } peers { permanent_uuid: "0d47f58aa3ee4663908c4bbe6d903b04" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 41679 } }
I20260809 02:19:57.802445 12414 heartbeater.cc:503] Master 127.15.75.190:38651 was elected leader, sending a full tablet report...
I20260809 02:19:57.802556 12441 ts_tablet_manager.cc:1428] T de04ab0897e943b4aba9dc9985677f24 P 408ec93995d343c9b2efdd1709e418f2: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20260809 02:19:57.802594 12440 ts_tablet_manager.cc:1428] T de04ab0897e943b4aba9dc9985677f24 P 25c56c5acf30469e983da9033bebfaa7: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 02:19:57.802958 12037 heartbeater.cc:503] Master 127.15.75.190:38651 was elected leader, sending a full tablet report...
I20260809 02:19:57.804530 12444 raft_consensus.cc:348] T de04ab0897e943b4aba9dc9985677f24 P 0d47f58aa3ee4663908c4bbe6d903b04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408ec93995d343c9b2efdd1709e418f2" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 37451 } } peers { permanent_uuid: "25c56c5acf30469e983da9033bebfaa7" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 34533 } } peers { permanent_uuid: "0d47f58aa3ee4663908c4bbe6d903b04" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 41679 } }
I20260809 02:19:57.804855 12444 raft_consensus.cc:374] T de04ab0897e943b4aba9dc9985677f24 P 0d47f58aa3ee4663908c4bbe6d903b04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:19:57.804926 12444 raft_consensus.cc:729] T de04ab0897e943b4aba9dc9985677f24 P 0d47f58aa3ee4663908c4bbe6d903b04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d47f58aa3ee4663908c4bbe6d903b04, State: Initialized, Role: FOLLOWER
I20260809 02:19:57.805339 12444 consensus_queue.cc:260] T de04ab0897e943b4aba9dc9985677f24 P 0d47f58aa3ee4663908c4bbe6d903b04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408ec93995d343c9b2efdd1709e418f2" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 37451 } } peers { permanent_uuid: "25c56c5acf30469e983da9033bebfaa7" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 34533 } } peers { permanent_uuid: "0d47f58aa3ee4663908c4bbe6d903b04" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 41679 } }
I20260809 02:19:57.805963 12217 heartbeater.cc:503] Master 127.15.75.190:38651 was elected leader, sending a full tablet report...
I20260809 02:19:57.806320 12444 ts_tablet_manager.cc:1428] T de04ab0897e943b4aba9dc9985677f24 P 0d47f58aa3ee4663908c4bbe6d903b04: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
W20260809 02:19:57.859840 12038 tablet.cc:2369] T de04ab0897e943b4aba9dc9985677f24 P 25c56c5acf30469e983da9033bebfaa7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:19:57.954254 12455 raft_consensus.cc:482] T de04ab0897e943b4aba9dc9985677f24 P 0d47f58aa3ee4663908c4bbe6d903b04 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:19:57.954541 12455 raft_consensus.cc:504] T de04ab0897e943b4aba9dc9985677f24 P 0d47f58aa3ee4663908c4bbe6d903b04 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408ec93995d343c9b2efdd1709e418f2" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 37451 } } peers { permanent_uuid: "25c56c5acf30469e983da9033bebfaa7" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 34533 } } peers { permanent_uuid: "0d47f58aa3ee4663908c4bbe6d903b04" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 41679 } }
I20260809 02:19:57.955917 12455 leader_election.cc:288] T de04ab0897e943b4aba9dc9985677f24 P 0d47f58aa3ee4663908c4bbe6d903b04 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 408ec93995d343c9b2efdd1709e418f2 (127.15.75.131:37451), 25c56c5acf30469e983da9033bebfaa7 (127.15.75.129:34533)
I20260809 02:19:57.960304 12433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:57.956299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.130:38273 (local address 127.15.75.131:37451)
0809 02:19:57.956431 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:57.956436 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:57.956455 (+ 19us) server_negotiation.cc:407] Connection header received
0809 02:19:57.956644 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:57.956648 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:57.956728 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:57.956814 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:19:57.957186 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:57.957825 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:57.959190 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:57.959744 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:57.959836 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:57.959874 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:57.960103 (+ 229us) server_negotiation.cc:299] Negotiation successful
0809 02:19:57.960159 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 02:19:57.960309 12219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:57.956187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.131:37451 (local address 127.15.75.130:38273)
0809 02:19:57.956348 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:57.956367 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 02:19:57.956490 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:19:57.956841 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:19:57.956843 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:19:57.956865 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:19:57.957059 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:57.957069 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:57.957943 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:57.957947 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:57.959053 (+ 1106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:57.959068 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:57.959865 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:57.959869 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:57.959970 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:57.959996 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 02:19:57.960080 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 02:19:57.960152 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56,"mutex_wait_us":1,"spinlock_wait_cycles":34048}
I20260809 02:19:57.960772 12457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:57.956187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.129:34533 (local address 127.15.75.130:51049)
0809 02:19:57.956817 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:57.956832 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:19:57.956955 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:19:57.957245 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:19:57.957248 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:19:57.957266 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:19:57.957396 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:57.957403 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:57.958366 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:57.958368 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:57.959423 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:57.959434 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:57.960258 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:57.960261 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:57.960361 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:57.960390 (+ 29us) client_negotiation.cc:769] Sending connection context
0809 02:19:57.960511 (+ 121us) client_negotiation.cc:240] Negotiation successful
0809 02:19:57.960593 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"mutex_wait_us":188,"thread_start_us":158,"threads_started":1}
I20260809 02:19:57.960809 12437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:57.956310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.130:51049 (local address 127.15.75.129:34533)
0809 02:19:57.956442 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:57.956448 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:57.956927 (+ 479us) server_negotiation.cc:407] Connection header received
0809 02:19:57.957072 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:57.957076 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:57.957143 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:57.957224 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:19:57.957561 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:57.958241 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:57.959582 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:57.960131 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:57.960257 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:57.960301 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:57.960539 (+ 238us) server_negotiation.cc:299] Negotiation successful
0809 02:19:57.960625 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 02:19:57.961125 12347 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de04ab0897e943b4aba9dc9985677f24" candidate_uuid: "0d47f58aa3ee4663908c4bbe6d903b04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "408ec93995d343c9b2efdd1709e418f2" is_pre_election: true
I20260809 02:19:57.961457 12347 raft_consensus.cc:2454] T de04ab0897e943b4aba9dc9985677f24 P 408ec93995d343c9b2efdd1709e418f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d47f58aa3ee4663908c4bbe6d903b04 in term 0.
I20260809 02:19:57.961501 11967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de04ab0897e943b4aba9dc9985677f24" candidate_uuid: "0d47f58aa3ee4663908c4bbe6d903b04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25c56c5acf30469e983da9033bebfaa7" is_pre_election: true
I20260809 02:19:57.961771 11967 raft_consensus.cc:2454] T de04ab0897e943b4aba9dc9985677f24 P 25c56c5acf30469e983da9033bebfaa7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d47f58aa3ee4663908c4bbe6d903b04 in term 0.
I20260809 02:19:57.961994 12058 leader_election.cc:302] T de04ab0897e943b4aba9dc9985677f24 P 0d47f58aa3ee4663908c4bbe6d903b04 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d47f58aa3ee4663908c4bbe6d903b04, 408ec93995d343c9b2efdd1709e418f2; no voters:
I20260809 02:19:57.962298 12455 raft_consensus.cc:2781] T de04ab0897e943b4aba9dc9985677f24 P 0d47f58aa3ee4663908c4bbe6d903b04 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:19:57.962393 12455 raft_consensus.cc:482] T de04ab0897e943b4aba9dc9985677f24 P 0d47f58aa3ee4663908c4bbe6d903b04 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:19:57.962438 12455 raft_consensus.cc:3037] T de04ab0897e943b4aba9dc9985677f24 P 0d47f58aa3ee4663908c4bbe6d903b04 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:19:57.963464 12455 raft_consensus.cc:504] T de04ab0897e943b4aba9dc9985677f24 P 0d47f58aa3ee4663908c4bbe6d903b04 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408ec93995d343c9b2efdd1709e418f2" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 37451 } } peers { permanent_uuid: "25c56c5acf30469e983da9033bebfaa7" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 34533 } } peers { permanent_uuid: "0d47f58aa3ee4663908c4bbe6d903b04" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 41679 } }
I20260809 02:19:57.963792 12455 leader_election.cc:288] T de04ab0897e943b4aba9dc9985677f24 P 0d47f58aa3ee4663908c4bbe6d903b04 [CANDIDATE]: Term 1 election: Requested vote from peers 408ec93995d343c9b2efdd1709e418f2 (127.15.75.131:37451), 25c56c5acf30469e983da9033bebfaa7 (127.15.75.129:34533)
I20260809 02:19:57.964071 12347 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de04ab0897e943b4aba9dc9985677f24" candidate_uuid: "0d47f58aa3ee4663908c4bbe6d903b04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "408ec93995d343c9b2efdd1709e418f2"
I20260809 02:19:57.964135 11967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de04ab0897e943b4aba9dc9985677f24" candidate_uuid: "0d47f58aa3ee4663908c4bbe6d903b04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25c56c5acf30469e983da9033bebfaa7"
I20260809 02:19:57.964232 11967 raft_consensus.cc:3037] T de04ab0897e943b4aba9dc9985677f24 P 25c56c5acf30469e983da9033bebfaa7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:19:57.964239 12347 raft_consensus.cc:3037] T de04ab0897e943b4aba9dc9985677f24 P 408ec93995d343c9b2efdd1709e418f2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:19:57.965299 11967 raft_consensus.cc:2454] T de04ab0897e943b4aba9dc9985677f24 P 25c56c5acf30469e983da9033bebfaa7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d47f58aa3ee4663908c4bbe6d903b04 in term 1.
I20260809 02:19:57.965389 12347 raft_consensus.cc:2454] T de04ab0897e943b4aba9dc9985677f24 P 408ec93995d343c9b2efdd1709e418f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d47f58aa3ee4663908c4bbe6d903b04 in term 1.
I20260809 02:19:57.965662 12060 leader_election.cc:302] T de04ab0897e943b4aba9dc9985677f24 P 0d47f58aa3ee4663908c4bbe6d903b04 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d47f58aa3ee4663908c4bbe6d903b04, 25c56c5acf30469e983da9033bebfaa7; no voters:
I20260809 02:19:57.965871 12455 raft_consensus.cc:2781] T de04ab0897e943b4aba9dc9985677f24 P 0d47f58aa3ee4663908c4bbe6d903b04 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:19:57.966065 12455 raft_consensus.cc:686] T de04ab0897e943b4aba9dc9985677f24 P 0d47f58aa3ee4663908c4bbe6d903b04 [term 1 LEADER]: Becoming Leader. State: Replica: 0d47f58aa3ee4663908c4bbe6d903b04, State: Running, Role: LEADER
I20260809 02:19:57.966418 12455 consensus_queue.cc:237] T de04ab0897e943b4aba9dc9985677f24 P 0d47f58aa3ee4663908c4bbe6d903b04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408ec93995d343c9b2efdd1709e418f2" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 37451 } } peers { permanent_uuid: "25c56c5acf30469e983da9033bebfaa7" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 34533 } } peers { permanent_uuid: "0d47f58aa3ee4663908c4bbe6d903b04" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 41679 } }
I20260809 02:19:57.969574 11769 catalog_manager.cc:5495] T de04ab0897e943b4aba9dc9985677f24 P 0d47f58aa3ee4663908c4bbe6d903b04 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d47f58aa3ee4663908c4bbe6d903b04 (127.15.75.130). New cstate: current_term: 1 leader_uuid: "0d47f58aa3ee4663908c4bbe6d903b04" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "408ec93995d343c9b2efdd1709e418f2" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 37451 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25c56c5acf30469e983da9033bebfaa7" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 34533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d47f58aa3ee4663908c4bbe6d903b04" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 41679 } health_report { overall_health: HEALTHY } } }
W20260809 02:19:57.982250 12415 tablet.cc:2369] T de04ab0897e943b4aba9dc9985677f24 P 408ec93995d343c9b2efdd1709e418f2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:19:58.014356 12438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:58.010796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38344 (local address 127.15.75.130:41679)
0809 02:19:58.010925 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:58.010928 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:58.010941 (+ 13us) server_negotiation.cc:407] Connection header received
0809 02:19:58.011009 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:58.011011 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:58.011064 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:58.011153 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:19:58.011389 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:58.011901 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:58.012802 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:58.013097 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:58.013246 (+ 149us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:58.013513 (+ 267us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:19:58.013915 (+ 402us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:19:58.014025 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:58.014175 (+ 150us) server_negotiation.cc:299] Negotiation successful
0809 02:19:58.014207 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 02:19:58.020428 12347 raft_consensus.cc:1261] T de04ab0897e943b4aba9dc9985677f24 P 408ec93995d343c9b2efdd1709e418f2 [term 1 FOLLOWER]: Refusing update from remote peer 0d47f58aa3ee4663908c4bbe6d903b04: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:19:58.020428 11967 raft_consensus.cc:1261] T de04ab0897e943b4aba9dc9985677f24 P 25c56c5acf30469e983da9033bebfaa7 [term 1 FOLLOWER]: Refusing update from remote peer 0d47f58aa3ee4663908c4bbe6d903b04: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:19:58.021194 12455 consensus_queue.cc:1035] T de04ab0897e943b4aba9dc9985677f24 P 0d47f58aa3ee4663908c4bbe6d903b04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25c56c5acf30469e983da9033bebfaa7" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 34533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:19:58.021365 12458 consensus_queue.cc:1035] T de04ab0897e943b4aba9dc9985677f24 P 0d47f58aa3ee4663908c4bbe6d903b04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "408ec93995d343c9b2efdd1709e418f2" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 37451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:19:58.029856 12463 mvcc.cc:205] Tried to move back new op lower bound from 7316447223884169216 to 7316447223672889344. Current Snapshot: MvccSnapshot[applied={T|T < 7316447223884169216}]
I20260809 02:19:58.039767 11765 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60092:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:19:58.039955 11765 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60092:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:19:58.042424 11765 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8905d472eeb24cc0a8b9c37dd4e8a701: Sending DeleteTablet for 3 replicas of tablet de04ab0897e943b4aba9dc9985677f24
I20260809 02:19:58.043238 12132 tablet_service.cc:1514] Processing DeleteTablet for tablet de04ab0897e943b4aba9dc9985677f24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:19:58 UTC) from {username='slave'} at 127.0.0.1:38330
I20260809 02:19:58.043813 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 11853
I20260809 02:19:58.043809 12312 tablet_service.cc:1514] Processing DeleteTablet for tablet de04ab0897e943b4aba9dc9985677f24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:19:58 UTC) from {username='slave'} at 127.0.0.1:60026
I20260809 02:19:58.043838 11939 tablet_service.cc:1514] Processing DeleteTablet for tablet de04ab0897e943b4aba9dc9985677f24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:19:58 UTC) from {username='slave'} at 127.0.0.1:48348
I20260809 02:19:58.044468 12472 tablet_replica.cc:330] T de04ab0897e943b4aba9dc9985677f24 P 0d47f58aa3ee4663908c4bbe6d903b04: stopping tablet replica
I20260809 02:19:58.044854 12472 raft_consensus.cc:2229] T de04ab0897e943b4aba9dc9985677f24 P 0d47f58aa3ee4663908c4bbe6d903b04 [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:19:58.045310 12472 raft_consensus.cc:2258] T de04ab0897e943b4aba9dc9985677f24 P 0d47f58aa3ee4663908c4bbe6d903b04 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:19:58.053361 12473 tablet_replica.cc:330] T de04ab0897e943b4aba9dc9985677f24 P 408ec93995d343c9b2efdd1709e418f2: stopping tablet replica
I20260809 02:19:58.053742 12473 raft_consensus.cc:2229] T de04ab0897e943b4aba9dc9985677f24 P 408ec93995d343c9b2efdd1709e418f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:19:58.054080 12473 raft_consensus.cc:2258] T de04ab0897e943b4aba9dc9985677f24 P 408ec93995d343c9b2efdd1709e418f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:19:58.055634 12473 ts_tablet_manager.cc:1905] T de04ab0897e943b4aba9dc9985677f24 P 408ec93995d343c9b2efdd1709e418f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:19:58.057735 12472 ts_tablet_manager.cc:1905] T de04ab0897e943b4aba9dc9985677f24 P 0d47f58aa3ee4663908c4bbe6d903b04: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:19:58.058436 12473 ts_tablet_manager.cc:1918] T de04ab0897e943b4aba9dc9985677f24 P 408ec93995d343c9b2efdd1709e418f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:19:58.058537 12473 log.cc:1196] T de04ab0897e943b4aba9dc9985677f24 P 408ec93995d343c9b2efdd1709e418f2: Deleting WAL directory at /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241741788589-15662-0/minicluster-data/ts-2/wal/wals/de04ab0897e943b4aba9dc9985677f24
I20260809 02:19:58.058864 12473 ts_tablet_manager.cc:1939] T de04ab0897e943b4aba9dc9985677f24 P 408ec93995d343c9b2efdd1709e418f2: Deleting consensus metadata
W20260809 02:19:58.060146 11749 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.75.129:34533 (error 108)
I20260809 02:19:58.060312 11747 catalog_manager.cc:4841] TS 408ec93995d343c9b2efdd1709e418f2 (127.15.75.131:37451): tablet de04ab0897e943b4aba9dc9985677f24 (table test-table [id=b52f77a893624d4d8df5df56a61a6d4a]) successfully deleted
I20260809 02:19:58.061378 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 12041
I20260809 02:19:58.061738 12436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:58.061417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.129:34533 (local address 127.0.0.1:48360)
0809 02:19:58.061585 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:58.061645 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.129:34533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
W20260809 02:19:58.071203 11749 catalog_manager.cc:4584] TS 25c56c5acf30469e983da9033bebfaa7 (127.15.75.129:34533): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet de04ab0897e943b4aba9dc9985677f24: Network error: Client connection negotiation failed: client connection to 127.15.75.129:34533: connect: Connection refused (error 111)
I20260809 02:19:58.082479 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 12222
I20260809 02:19:58.084303 12436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:58.082627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.130:41679 (local address 127.0.0.1:38348)
0809 02:19:58.084109 (+ 1482us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:58.084184 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.130:41679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1358}
I20260809 02:19:58.091519 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 11727
2026-08-09T02:19:58Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1187 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:19:58 dist-test-slave-s11t krb5kdc[12480](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:19:58 dist-test-slave-s11t krb5kdc[12480](info): set up 2 sockets
Aug 09 02:19:58 dist-test-slave-s11t krb5kdc[12480](info): commencing operation
krb5kdc: starting...
W20260809 02:19:59.322451 15662 mini_kdc.cc:121] Time spent starting KDC: real 1.202s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Aug 09 02:19:59 dist-test-slave-s11t krb5kdc[12480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241999, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:19:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:19:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.75.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab.
Entry for principal kudu/127.15.75.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab.
I20260809 02:19:59.445760 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:42661
--webserver_interface=127.15.75.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab
--principal=kudu/127.15.75.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:33559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.75.190:42661
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:19:59.556892 12498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:19:59.557116 12498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:19:59.557154 12498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:19:59.561213 12498 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:19:59.561259 12498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:19:59.561280 12498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:19:59.561334 12498 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:19:59.561362 12498 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:19:59.566354 12498 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:33559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.75.190:42661
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab
--principal=kudu/127.15.75.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:42661
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.75.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12498
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:19:59.567268 12498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:19:59.568223 12498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:19:59.576541 12512 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:19:59.576593 12508 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:19:59.576583 12498 server_base.cc:1034] running on GCE node
W20260809 02:19:59.576723 12510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:19:59.577409 12498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:19:59.578351 12498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:19:59.579507 12498 hybrid_clock.cc:648] HybridClock initialized: now 1786241999579483 us; error 38 us; skew 500 ppm
Aug 09 02:19:59 dist-test-slave-s11t krb5kdc[12480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241999, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:19:59.582268 12498 init.cc:377] Logged in from keytab as kudu/127.15.75.190@KRBTEST.COM (short username kudu)
I20260809 02:19:59.583146 12498 webserver.cc:460] Webserver started at http://127.15.75.190:43453/ using document root <none> and password file <none>
I20260809 02:19:59.583676 12498 fs_manager.cc:359] Metadata directory not provided
I20260809 02:19:59.583735 12498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:19:59.583968 12498 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:19:59.586012 12498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/minicluster-data/master-0/data/instance:
uuid: "df9aab2447d044a1a930354ef70ecc03"
format_stamp: "Formatted at 2026-08-09 02:19:59 on dist-test-slave-s11t"
server_key: "a545e286f4fb0f94b324bd9f0d404ea3"
server_key_iv: "90d49d44c30243fecb9463947a84156c"
server_key_version: "encryptionkey@0"
I20260809 02:19:59.586591 12498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/minicluster-data/master-0/wal/instance:
uuid: "df9aab2447d044a1a930354ef70ecc03"
format_stamp: "Formatted at 2026-08-09 02:19:59 on dist-test-slave-s11t"
server_key: "a545e286f4fb0f94b324bd9f0d404ea3"
server_key_iv: "90d49d44c30243fecb9463947a84156c"
server_key_version: "encryptionkey@0"
I20260809 02:19:59.590186 12498 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20260809 02:19:59.592428 12519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:19:59.593631 12498 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 02:19:59.593771 12498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/minicluster-data/master-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/minicluster-data/master-0/wal
uuid: "df9aab2447d044a1a930354ef70ecc03"
format_stamp: "Formatted at 2026-08-09 02:19:59 on dist-test-slave-s11t"
server_key: "a545e286f4fb0f94b324bd9f0d404ea3"
server_key_iv: "90d49d44c30243fecb9463947a84156c"
server_key_version: "encryptionkey@0"
I20260809 02:19:59.593878 12498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:19:59.609999 12498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:19:59.612679 12498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:19:59.612916 12498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:19:59.619704 12498 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.190:42661
I20260809 02:19:59.619753 12598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.190:42661 every 8 connection(s)
I20260809 02:19:59.620688 12498 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
I20260809 02:19:59.621681 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 12498
I20260809 02:19:59.621817 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/minicluster-data/master-0/wal/instance
I20260809 02:19:59.622110 15662 external_mini_cluster.cc:1428] Setting key 8f6fc8acded125be990e97b5276a6489
I20260809 02:19:59.623910 12600 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:19:59.630697 12600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df9aab2447d044a1a930354ef70ecc03: Bootstrap starting.
Aug 09 02:19:59 dist-test-slave-s11t krb5kdc[12480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241999, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:19:59.633430 12600 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P df9aab2447d044a1a930354ef70ecc03: Neither blocks nor log segments found. Creating new log.
I20260809 02:19:59.634330 12600 log.cc:824] T 00000000000000000000000000000000 P df9aab2447d044a1a930354ef70ecc03: Log is configured to *not* fsync() on all Append() calls
I20260809 02:19:59.636621 12600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df9aab2447d044a1a930354ef70ecc03: No bootstrap required, opened a new log
I20260809 02:19:59.637387 12605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:59.623514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53264 (local address 127.15.75.190:42661)
0809 02:19:59.623952 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:59.623974 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:59.624017 (+ 43us) server_negotiation.cc:407] Connection header received
0809 02:19:59.624868 (+ 851us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:59.624891 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:59.625282 (+ 391us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:59.625650 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:59.626310 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:59.627844 (+ 1534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:59.628728 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:59.629425 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:59.629601 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:59.632141 (+ 2540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:19:59.632147 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:19:59.632162 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:19:59.632192 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:19:59.634201 (+ 2009us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:59.634674 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:59.634683 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:59.634690 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:59.634840 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:59.635190 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:59.635194 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:59.635195 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:59.635681 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:19:59.635872 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:59.636127 (+ 255us) server_negotiation.cc:299] Negotiation successful
0809 02:19:59.636430 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":124,"threads_started":1}
I20260809 02:19:59.640141 12600 raft_consensus.cc:348] T 00000000000000000000000000000000 P df9aab2447d044a1a930354ef70ecc03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df9aab2447d044a1a930354ef70ecc03" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 42661 } }
I20260809 02:19:59.640349 12600 raft_consensus.cc:374] T 00000000000000000000000000000000 P df9aab2447d044a1a930354ef70ecc03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:19:59.640384 12600 raft_consensus.cc:729] T 00000000000000000000000000000000 P df9aab2447d044a1a930354ef70ecc03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df9aab2447d044a1a930354ef70ecc03, State: Initialized, Role: FOLLOWER
I20260809 02:19:59.640839 12600 consensus_queue.cc:260] T 00000000000000000000000000000000 P df9aab2447d044a1a930354ef70ecc03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df9aab2447d044a1a930354ef70ecc03" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 42661 } }
I20260809 02:19:59.640966 12600 raft_consensus.cc:388] T 00000000000000000000000000000000 P df9aab2447d044a1a930354ef70ecc03 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:19:59.641014 12600 raft_consensus.cc:482] T 00000000000000000000000000000000 P df9aab2447d044a1a930354ef70ecc03 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:19:59.641086 12600 raft_consensus.cc:3037] T 00000000000000000000000000000000 P df9aab2447d044a1a930354ef70ecc03 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:19:59.642024 12600 raft_consensus.cc:504] T 00000000000000000000000000000000 P df9aab2447d044a1a930354ef70ecc03 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df9aab2447d044a1a930354ef70ecc03" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 42661 } }
I20260809 02:19:59.642390 12600 leader_election.cc:302] T 00000000000000000000000000000000 P df9aab2447d044a1a930354ef70ecc03 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: df9aab2447d044a1a930354ef70ecc03; no voters:
I20260809 02:19:59.642746 12600 leader_election.cc:288] T 00000000000000000000000000000000 P df9aab2447d044a1a930354ef70ecc03 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:19:59.642868 12608 raft_consensus.cc:2781] T 00000000000000000000000000000000 P df9aab2447d044a1a930354ef70ecc03 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:19:59.643123 12608 raft_consensus.cc:686] T 00000000000000000000000000000000 P df9aab2447d044a1a930354ef70ecc03 [term 1 LEADER]: Becoming Leader. State: Replica: df9aab2447d044a1a930354ef70ecc03, State: Running, Role: LEADER
I20260809 02:19:59.643455 12608 consensus_queue.cc:237] T 00000000000000000000000000000000 P df9aab2447d044a1a930354ef70ecc03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df9aab2447d044a1a930354ef70ecc03" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 42661 } }
I20260809 02:19:59.643947 12600 sys_catalog.cc:564] T 00000000000000000000000000000000 P df9aab2447d044a1a930354ef70ecc03 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:19:59.645182 12611 sys_catalog.cc:455] T 00000000000000000000000000000000 P df9aab2447d044a1a930354ef70ecc03 [sys.catalog]: SysCatalogTable state changed. Reason: New leader df9aab2447d044a1a930354ef70ecc03. Latest consensus state: current_term: 1 leader_uuid: "df9aab2447d044a1a930354ef70ecc03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df9aab2447d044a1a930354ef70ecc03" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 42661 } } }
I20260809 02:19:59.645183 12610 sys_catalog.cc:455] T 00000000000000000000000000000000 P df9aab2447d044a1a930354ef70ecc03 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "df9aab2447d044a1a930354ef70ecc03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df9aab2447d044a1a930354ef70ecc03" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 42661 } } }
I20260809 02:19:59.645344 12610 sys_catalog.cc:458] T 00000000000000000000000000000000 P df9aab2447d044a1a930354ef70ecc03 [sys.catalog]: This master's current role is: LEADER
I20260809 02:19:59.645344 12611 sys_catalog.cc:458] T 00000000000000000000000000000000 P df9aab2447d044a1a930354ef70ecc03 [sys.catalog]: This master's current role is: LEADER
I20260809 02:19:59.645869 12625 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:19:59.648778 12625 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:19:59.654006 12625 catalog_manager.cc:1347] Generated new cluster ID: 73d862a1f0b2404d83d005497a973cba
I20260809 02:19:59.654068 12625 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:19:59.672380 12625 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:19:59.673332 12625 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:19:59.682492 12625 catalog_manager.cc:5868] T 00000000000000000000000000000000 P df9aab2447d044a1a930354ef70ecc03: Generated new TSK 0
I20260809 02:19:59.683210 12625 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.75.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab.
Entry for principal kudu/127.15.75.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab.
I20260809 02:19:59.715395 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.129:0
--local_ip_for_outbound_sockets=127.15.75.129
--webserver_interface=127.15.75.129
--webserver_port=0
--tserver_master_addrs=127.15.75.190:42661
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab
--principal=kudu/127.15.75.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:33559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:19:59.826016 12642 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:19:59.826236 12642 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:19:59.826277 12642 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:19:59.830161 12642 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:19:59.830209 12642 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:19:59.830263 12642 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.129
I20260809 02:19:59.835445 12642 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:33559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.129.keytab
--principal=kudu/127.15.75.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.75.129
--webserver_port=0
--tserver_master_addrs=127.15.75.190:42661
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12642
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.129
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:19:59.836391 12642 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:19:59.837411 12642 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:19:59.846947 12649 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:19:59.846961 12657 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:19:59.846961 12648 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:19:59.847208 12642 server_base.cc:1034] running on GCE node
I20260809 02:19:59.847723 12642 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:19:59.848315 12642 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:19:59.849504 12642 hybrid_clock.cc:648] HybridClock initialized: now 1786241999849474 us; error 41 us; skew 500 ppm
Aug 09 02:19:59 dist-test-slave-s11t krb5kdc[12480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241999, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:19:59.852766 12642 init.cc:377] Logged in from keytab as kudu/127.15.75.129@KRBTEST.COM (short username kudu)
I20260809 02:19:59.853668 12642 webserver.cc:460] Webserver started at http://127.15.75.129:37209/ using document root <none> and password file <none>
I20260809 02:19:59.854211 12642 fs_manager.cc:359] Metadata directory not provided
I20260809 02:19:59.854275 12642 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:19:59.854449 12642 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:19:59.856484 12642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/minicluster-data/ts-0/data/instance:
uuid: "7b2850c44df641f0a9a4b05a1049f190"
format_stamp: "Formatted at 2026-08-09 02:19:59 on dist-test-slave-s11t"
server_key: "1c350fa12a42203c06970a195839857d"
server_key_iv: "e066c9ac5a7be3f340b70a76a535866e"
server_key_version: "encryptionkey@0"
I20260809 02:19:59.857120 12642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/minicluster-data/ts-0/wal/instance:
uuid: "7b2850c44df641f0a9a4b05a1049f190"
format_stamp: "Formatted at 2026-08-09 02:19:59 on dist-test-slave-s11t"
server_key: "1c350fa12a42203c06970a195839857d"
server_key_iv: "e066c9ac5a7be3f340b70a76a535866e"
server_key_version: "encryptionkey@0"
I20260809 02:19:59.860934 12642 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 02:19:59.863343 12664 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:19:59.864581 12642 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 02:19:59.864764 12642 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/minicluster-data/ts-0/wal
uuid: "7b2850c44df641f0a9a4b05a1049f190"
format_stamp: "Formatted at 2026-08-09 02:19:59 on dist-test-slave-s11t"
server_key: "1c350fa12a42203c06970a195839857d"
server_key_iv: "e066c9ac5a7be3f340b70a76a535866e"
server_key_version: "encryptionkey@0"
I20260809 02:19:59.864881 12642 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:19:59.893759 12642 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:19:59.896651 12642 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:19:59.896891 12642 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:19:59.897415 12642 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:19:59.898273 12642 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:19:59.898319 12642 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:19:59.898358 12642 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:19:59.898373 12642 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:19:59.907392 12642 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.129:35967
I20260809 02:19:59.907464 12840 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.129:35967 every 8 connection(s)
I20260809 02:19:59.908275 12642 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
I20260809 02:19:59.911975 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 12642
I20260809 02:19:59.912135 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/minicluster-data/ts-0/wal/instance
I20260809 02:19:59.912439 15662 external_mini_cluster.cc:1428] Setting key 361f258b00680a162cbd20337213af57
Aug 09 02:19:59 dist-test-slave-s11t krb5kdc[12480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241999, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.129@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:19:59.923353 12605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:59.910017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.129:50687 (local address 127.15.75.190:42661)
0809 02:19:59.910147 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:59.910151 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:59.911330 (+ 1179us) server_negotiation.cc:407] Connection header received
0809 02:19:59.912290 (+ 960us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:59.912294 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:59.912359 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:59.912470 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:59.913802 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:59.914399 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:59.916313 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:59.916753 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:59.916876 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:59.919309 (+ 2433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:19:59.919314 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:19:59.919321 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:19:59.919355 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:19:59.921135 (+ 1780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:59.921697 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:59.921702 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:59.921706 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:59.921803 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:59.922146 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:59.922151 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:59.922154 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:59.922415 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:19:59.922529 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:59.923001 (+ 472us) server_negotiation.cc:299] Negotiation successful
0809 02:19:59.923146 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
WARNING: no policy specified for kudu/127.15.75.130@KRBTEST.COM; defaulting to no policy
I20260809 02:19:59.924057 12844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:59.910304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:42661 (local address 127.15.75.129:50687)
0809 02:19:59.911173 (+ 869us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:59.911214 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 02:19:59.912021 (+ 807us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:19:59.912644 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:19:59.912651 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:19:59.913106 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:19:59.913641 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:59.913664 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:59.914535 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:59.914542 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:59.916162 (+ 1620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:59.916178 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:59.916916 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:59.916919 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:59.917056 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:59.917645 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:19:59.917661 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:19:59.919108 (+ 1447us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:19:59.921254 (+ 2146us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:19:59.921258 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:19:59.921269 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:19:59.921583 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:19:59.921911 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:19:59.921913 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:19:59.921914 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:19:59.922049 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:19:59.922527 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:19:59.922532 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:19:59.922753 (+ 221us) client_negotiation.cc:769] Sending connection context
0809 02:19:59.922904 (+ 151us) client_negotiation.cc:240] Negotiation successful
0809 02:19:59.923137 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":688,"thread_start_us":549,"threads_started":1}
I20260809 02:19:59.925312 12842 heartbeater.cc:347] Connected to a master server at 127.15.75.190:42661
I20260809 02:19:59.925511 12842 heartbeater.cc:464] Registering TS with master...
I20260809 02:19:59.926017 12842 heartbeater.cc:511] Master 127.15.75.190:42661 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.130@KRBTEST.COM" created.
I20260809 02:19:59.927677 12549 ts_manager.cc:194] Registered new tserver with Master: 7b2850c44df641f0a9a4b05a1049f190 (127.15.75.129:35967)
I20260809 02:19:59.929103 12549 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.75.129@KRBTEST.COM'} at 127.15.75.129:50687
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab.
Entry for principal kudu/127.15.75.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab.
I20260809 02:19:59.941841 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.130:0
--local_ip_for_outbound_sockets=127.15.75.130
--webserver_interface=127.15.75.130
--webserver_port=0
--tserver_master_addrs=127.15.75.190:42661
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab
--principal=kudu/127.15.75.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:33559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:20:00.055436 12847 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:20:00.055666 12847 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:20:00.055706 12847 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:20:00.059746 12847 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:20:00.059799 12847 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:20:00.059906 12847 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.130
I20260809 02:20:00.065328 12847 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:33559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.130.keytab
--principal=kudu/127.15.75.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.75.130
--webserver_port=0
--tserver_master_addrs=127.15.75.190:42661
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12847
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.130
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:20:00.066351 12847 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:20:00.067408 12847 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:20:00.077118 12856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:20:00.077229 12847 server_base.cc:1034] running on GCE node
W20260809 02:20:00.077136 12854 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:20:00.077118 12858 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:20:00.077740 12847 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:20:00.078348 12847 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:20:00.079535 12847 hybrid_clock.cc:648] HybridClock initialized: now 1786242000079525 us; error 30 us; skew 500 ppm
Aug 09 02:20:00 dist-test-slave-s11t krb5kdc[12480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786242000, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:20:00.082295 12847 init.cc:377] Logged in from keytab as kudu/127.15.75.130@KRBTEST.COM (short username kudu)
I20260809 02:20:00.083124 12847 webserver.cc:460] Webserver started at http://127.15.75.130:34285/ using document root <none> and password file <none>
I20260809 02:20:00.083678 12847 fs_manager.cc:359] Metadata directory not provided
I20260809 02:20:00.083766 12847 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:20:00.084010 12847 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:20:00.086027 12847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/minicluster-data/ts-1/data/instance:
uuid: "c8ef9dacda574f9782106ea90e7986eb"
format_stamp: "Formatted at 2026-08-09 02:20:00 on dist-test-slave-s11t"
server_key: "35336f3625702b8eb5115308030ceaa5"
server_key_iv: "c31b85982a5194efc496cafb0e17bd6a"
server_key_version: "encryptionkey@0"
I20260809 02:20:00.086586 12847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/minicluster-data/ts-1/wal/instance:
uuid: "c8ef9dacda574f9782106ea90e7986eb"
format_stamp: "Formatted at 2026-08-09 02:20:00 on dist-test-slave-s11t"
server_key: "35336f3625702b8eb5115308030ceaa5"
server_key_iv: "c31b85982a5194efc496cafb0e17bd6a"
server_key_version: "encryptionkey@0"
I20260809 02:20:00.090358 12847 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 02:20:00.092664 12867 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:20:00.093919 12847 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 02:20:00.094022 12847 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/minicluster-data/ts-1/wal
uuid: "c8ef9dacda574f9782106ea90e7986eb"
format_stamp: "Formatted at 2026-08-09 02:20:00 on dist-test-slave-s11t"
server_key: "35336f3625702b8eb5115308030ceaa5"
server_key_iv: "c31b85982a5194efc496cafb0e17bd6a"
server_key_version: "encryptionkey@0"
I20260809 02:20:00.094149 12847 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:20:00.117398 12847 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:20:00.120186 12847 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:20:00.120376 12847 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:20:00.121016 12847 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:20:00.121819 12847 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:20:00.121865 12847 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:20:00.121903 12847 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:20:00.121919 12847 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:20:00.130986 12847 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.130:36745
I20260809 02:20:00.131022 13040 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.130:36745 every 8 connection(s)
I20260809 02:20:00.131819 12847 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
I20260809 02:20:00.138501 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 12847
I20260809 02:20:00.138628 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/minicluster-data/ts-1/wal/instance
I20260809 02:20:00.138947 15662 external_mini_cluster.cc:1428] Setting key 1f19451c0f5a01a49f3b79222926c08f
Aug 09 02:20:00 dist-test-slave-s11t krb5kdc[12480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786242000, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.130@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:20:00.147060 12605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:00.133599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.130:54581 (local address 127.15.75.190:42661)
0809 02:20:00.133732 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:00.133737 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:00.134604 (+ 867us) server_negotiation.cc:407] Connection header received
0809 02:20:00.136008 (+ 1404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:00.136012 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:00.136070 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:00.136176 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:00.137398 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:00.137949 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:00.139561 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:00.139870 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:00.139978 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:00.142976 (+ 2998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:20:00.142979 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:20:00.142981 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:20:00.143005 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:20:00.144608 (+ 1603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:00.145217 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:00.145223 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:00.145225 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:00.145321 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:00.145761 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:00.145767 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:00.145771 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:00.146036 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:20:00.146152 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:00.146656 (+ 504us) server_negotiation.cc:299] Negotiation successful
0809 02:20:00.146819 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 02:20:00.147599 13043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:00.134019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:42661 (local address 127.15.75.130:54581)
0809 02:20:00.134461 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0809 02:20:00.134502 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 02:20:00.135804 (+ 1302us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:20:00.136366 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:20:00.136373 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:20:00.136773 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:20:00.137223 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:20:00.137236 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:20:00.138080 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:20:00.138088 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:20:00.139404 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:20:00.139415 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:20:00.140017 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:20:00.140020 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:20:00.140206 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:00.141104 (+ 898us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:20:00.141129 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:20:00.142799 (+ 1670us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:20:00.144731 (+ 1932us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:20:00.144737 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:20:00.144752 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:20:00.145095 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:20:00.145434 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:20:00.145437 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:20:00.145439 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:20:00.145579 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:20:00.146147 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:20:00.146151 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:20:00.146391 (+ 240us) client_negotiation.cc:769] Sending connection context
0809 02:20:00.146531 (+ 140us) client_negotiation.cc:240] Negotiation successful
0809 02:20:00.146761 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":122,"threads_started":1}
I20260809 02:20:00.148741 13041 heartbeater.cc:347] Connected to a master server at 127.15.75.190:42661
I20260809 02:20:00.148969 13041 heartbeater.cc:464] Registering TS with master...
I20260809 02:20:00.149443 13041 heartbeater.cc:511] Master 127.15.75.190:42661 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.15.75.131@KRBTEST.COM; defaulting to no policy
I20260809 02:20:00.150473 12549 ts_manager.cc:194] Registered new tserver with Master: c8ef9dacda574f9782106ea90e7986eb (127.15.75.130:36745)
I20260809 02:20:00.151146 12549 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.75.130@KRBTEST.COM'} at 127.15.75.130:54581
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab.
Entry for principal kudu/127.15.75.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab.
I20260809 02:20:00.168057 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.131:0
--local_ip_for_outbound_sockets=127.15.75.131
--webserver_interface=127.15.75.131
--webserver_port=0
--tserver_master_addrs=127.15.75.190:42661
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab
--principal=kudu/127.15.75.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:33559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:20:00.283545 13049 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:20:00.283780 13049 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:20:00.283862 13049 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:20:00.287969 13049 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:20:00.288020 13049 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:20:00.288076 13049 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.131
I20260809 02:20:00.293507 13049 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:33559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.131.keytab
--principal=kudu/127.15.75.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.75.131
--webserver_port=0
--tserver_master_addrs=127.15.75.190:42661
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13049
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.131
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:20:00.294487 13049 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:20:00.295434 13049 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:20:00.305114 13058 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:20:00.305171 13060 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:20:00.305312 13049 server_base.cc:1034] running on GCE node
W20260809 02:20:00.305114 13062 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:20:00.305879 13049 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:20:00.306505 13049 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:20:00.307703 13049 hybrid_clock.cc:648] HybridClock initialized: now 1786242000307683 us; error 29 us; skew 500 ppm
Aug 09 02:20:00 dist-test-slave-s11t krb5kdc[12480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786242000, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:20:00.310566 13049 init.cc:377] Logged in from keytab as kudu/127.15.75.131@KRBTEST.COM (short username kudu)
I20260809 02:20:00.311486 13049 webserver.cc:460] Webserver started at http://127.15.75.131:36769/ using document root <none> and password file <none>
I20260809 02:20:00.312048 13049 fs_manager.cc:359] Metadata directory not provided
I20260809 02:20:00.312142 13049 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:20:00.312384 13049 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:20:00.314482 13049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/minicluster-data/ts-2/data/instance:
uuid: "1299c2ccb7fc46bda5d4ef0a5e10033b"
format_stamp: "Formatted at 2026-08-09 02:20:00 on dist-test-slave-s11t"
server_key: "90319c5a5ebc24ae794276f7fc44c9e4"
server_key_iv: "ca9f1a997795c6d01980a0480cb784eb"
server_key_version: "encryptionkey@0"
I20260809 02:20:00.315056 13049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/minicluster-data/ts-2/wal/instance:
uuid: "1299c2ccb7fc46bda5d4ef0a5e10033b"
format_stamp: "Formatted at 2026-08-09 02:20:00 on dist-test-slave-s11t"
server_key: "90319c5a5ebc24ae794276f7fc44c9e4"
server_key_iv: "ca9f1a997795c6d01980a0480cb784eb"
server_key_version: "encryptionkey@0"
I20260809 02:20:00.318725 13049 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 02:20:00.321086 13072 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:20:00.322278 13049 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 02:20:00.322422 13049 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/minicluster-data/ts-2/wal
uuid: "1299c2ccb7fc46bda5d4ef0a5e10033b"
format_stamp: "Formatted at 2026-08-09 02:20:00 on dist-test-slave-s11t"
server_key: "90319c5a5ebc24ae794276f7fc44c9e4"
server_key_iv: "ca9f1a997795c6d01980a0480cb784eb"
server_key_version: "encryptionkey@0"
I20260809 02:20:00.322527 13049 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:20:00.342907 13049 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:20:00.345922 13049 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:20:00.346127 13049 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:20:00.346712 13049 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:20:00.347558 13049 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:20:00.347661 13049 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:20:00.347760 13049 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:20:00.347820 13049 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:20:00.357023 13049 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.131:43159
I20260809 02:20:00.357054 13229 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.131:43159 every 8 connection(s)
I20260809 02:20:00.357885 13049 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
I20260809 02:20:00.359813 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 13049
I20260809 02:20:00.359949 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/minicluster-data/ts-2/wal/instance
I20260809 02:20:00.360272 15662 external_mini_cluster.cc:1428] Setting key ba1bb67074960e8453685cddd66ee3ce
Aug 09 02:20:00 dist-test-slave-s11t krb5kdc[12480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786242000, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.131@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:20:00.371425 12605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:00.359633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.131:39257 (local address 127.15.75.190:42661)
0809 02:20:00.359789 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:00.359794 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:00.360745 (+ 951us) server_negotiation.cc:407] Connection header received
0809 02:20:00.362461 (+ 1716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:00.362467 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:00.362528 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:00.362610 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:00.363744 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:00.364304 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:00.365497 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:00.365714 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:00.365803 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:00.368295 (+ 2492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:20:00.368298 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:20:00.368300 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:20:00.368324 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:20:00.369649 (+ 1325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:00.370105 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:00.370108 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:00.370109 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:00.370185 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:00.370483 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:00.370485 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:00.370486 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:00.370654 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:20:00.370729 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:00.371202 (+ 473us) server_negotiation.cc:299] Negotiation successful
0809 02:20:00.371309 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 02:20:00.372121 13233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:00.360068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:42661 (local address 127.15.75.131:39257)
0809 02:20:00.360562 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0809 02:20:00.360620 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 02:20:00.362188 (+ 1568us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:20:00.362829 (+ 641us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:20:00.362836 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:20:00.363161 (+ 325us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:20:00.363610 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:20:00.363623 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:20:00.364412 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:20:00.364414 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:20:00.365383 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:20:00.365393 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:20:00.365863 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:20:00.365866 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:20:00.366006 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:00.366535 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:20:00.366551 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:20:00.368106 (+ 1555us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:20:00.369759 (+ 1653us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:20:00.369763 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:20:00.369774 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:20:00.370015 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:20:00.370265 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:20:00.370267 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:20:00.370268 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:20:00.370404 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:20:00.370738 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:20:00.370742 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:20:00.370959 (+ 217us) client_negotiation.cc:769] Sending connection context
0809 02:20:00.371108 (+ 149us) client_negotiation.cc:240] Negotiation successful
0809 02:20:00.371346 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":107,"threads_started":1}
I20260809 02:20:00.373319 13230 heartbeater.cc:347] Connected to a master server at 127.15.75.190:42661
I20260809 02:20:00.373564 13230 heartbeater.cc:464] Registering TS with master...
I20260809 02:20:00.374156 13230 heartbeater.cc:511] Master 127.15.75.190:42661 requested a full tablet report, sending...
I20260809 02:20:00.375180 12549 ts_manager.cc:194] Registered new tserver with Master: 1299c2ccb7fc46bda5d4ef0a5e10033b (127.15.75.131:43159)
I20260809 02:20:00.375761 12549 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.75.131@KRBTEST.COM'} at 127.15.75.131:39257
I20260809 02:20:00.385882 15662 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 02:20:00.395401 12605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:00.387854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53272 (local address 127.15.75.190:42661)
0809 02:20:00.388001 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:00.388005 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:00.388242 (+ 237us) server_negotiation.cc:407] Connection header received
0809 02:20:00.388367 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:00.388374 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:00.388455 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:00.388536 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:00.389483 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:00.389969 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:00.390759 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:00.391009 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:00.391095 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:00.392299 (+ 1204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:20:00.392302 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:20:00.392304 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:20:00.392325 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:20:00.393735 (+ 1410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:00.394156 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:00.394160 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:00.394161 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:00.394244 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:00.394533 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:00.394536 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:00.394538 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:00.394748 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:20:00.394894 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:00.395118 (+ 224us) server_negotiation.cc:299] Negotiation successful
0809 02:20:00.395236 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 02:20:00.398317 12549 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53272:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:20:00.401232 12549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:20:00.415896 13246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:00.409840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.129:35967 (local address 127.0.0.1:45140)
0809 02:20:00.410501 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0809 02:20:00.410550 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 02:20:00.410677 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:20:00.411728 (+ 1051us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:20:00.411731 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:20:00.411767 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:20:00.411937 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:20:00.411944 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:20:00.413170 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:20:00.413175 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:20:00.414455 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:20:00.414465 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:20:00.415374 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:20:00.415378 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:20:00.415483 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:00.415520 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 02:20:00.415628 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 02:20:00.415745 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"mutex_wait_us":10,"spinlock_wait_cycles":3840,"thread_start_us":70,"threads_started":1}
I20260809 02:20:00.415923 13252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:00.409964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45140 (local address 127.15.75.129:35967)
0809 02:20:00.410475 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:00.411064 (+ 589us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:00.411098 (+ 34us) server_negotiation.cc:407] Connection header received
0809 02:20:00.411346 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:00.411353 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:00.411561 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:00.412092 (+ 531us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:20:00.412242 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:00.413039 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:00.414584 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:00.415249 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:00.415361 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:00.415504 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:00.415669 (+ 165us) server_negotiation.cc:299] Negotiation successful
0809 02:20:00.415768 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":418,"thread_start_us":153,"threads_started":1}
I20260809 02:20:00.416638 13253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:00.410556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.130:36745 (local address 127.0.0.1:41694)
0809 02:20:00.411029 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0809 02:20:00.411045 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 02:20:00.411118 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:20:00.411933 (+ 815us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:20:00.411936 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:20:00.411951 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:20:00.412048 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:20:00.412054 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:20:00.413463 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:20:00.413465 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:20:00.414974 (+ 1509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:20:00.414980 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:20:00.416218 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:20:00.416222 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:20:00.416325 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:00.416351 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 02:20:00.416423 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 02:20:00.416487 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":422,"mutex_wait_us":174,"spinlock_wait_cycles":3456,"thread_start_us":87,"threads_started":1}
I20260809 02:20:00.416991 13254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:00.410620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34350 (local address 127.15.75.131:43159)
0809 02:20:00.411339 (+ 719us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:00.411347 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:00.411365 (+ 18us) server_negotiation.cc:407] Connection header received
0809 02:20:00.411422 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:00.411426 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:00.411560 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:00.411722 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:20:00.412039 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:00.413288 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:00.415104 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:00.415957 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:00.416141 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:00.416238 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:00.416736 (+ 498us) server_negotiation.cc:299] Negotiation successful
0809 02:20:00.416837 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":633,"thread_start_us":156,"threads_started":1}
I20260809 02:20:00.417243 13250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:00.409894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.131:43159 (local address 127.0.0.1:34350)
0809 02:20:00.411001 (+ 1107us) negotiation.cc:107] Waiting for socket to connect
0809 02:20:00.411015 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:20:00.411109 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:20:00.411724 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:20:00.411730 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:20:00.411763 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:20:00.411925 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:20:00.411931 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:20:00.413454 (+ 1523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:20:00.413457 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:20:00.414969 (+ 1512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:20:00.414980 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:20:00.416385 (+ 1405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:20:00.416392 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:20:00.416476 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:00.416495 (+ 19us) client_negotiation.cc:769] Sending connection context
0809 02:20:00.416556 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 02:20:00.416608 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1041,"mutex_wait_us":241,"spinlock_wait_cycles":3840,"thread_start_us":98,"threads_started":1}
I20260809 02:20:00.417632 13255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:00.410718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41694 (local address 127.15.75.130:36745)
0809 02:20:00.411380 (+ 662us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:00.411392 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:00.411415 (+ 23us) server_negotiation.cc:407] Connection header received
0809 02:20:00.411479 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:00.411484 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:00.411628 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:00.411766 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:20:00.412174 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:00.413322 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:00.415126 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:00.415903 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:00.416015 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:00.416159 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:00.417437 (+ 1278us) server_negotiation.cc:299] Negotiation successful
0809 02:20:00.417530 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":544,"thread_start_us":160,"threads_started":1}
I20260809 02:20:00.419445 12747 tablet_service.cc:1467] Processing CreateTablet for tablet f99214a6b0fc4838bda811f10c0080c7 (DEFAULT_TABLE table=test-table [id=7a58484fa14d480d94ebfccfa131a21b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:20:00.419625 13135 tablet_service.cc:1467] Processing CreateTablet for tablet f99214a6b0fc4838bda811f10c0080c7 (DEFAULT_TABLE table=test-table [id=7a58484fa14d480d94ebfccfa131a21b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:20:00.420496 13135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f99214a6b0fc4838bda811f10c0080c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:20:00.420171 12942 tablet_service.cc:1467] Processing CreateTablet for tablet f99214a6b0fc4838bda811f10c0080c7 (DEFAULT_TABLE table=test-table [id=7a58484fa14d480d94ebfccfa131a21b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:20:00.420557 12747 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f99214a6b0fc4838bda811f10c0080c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:20:00.421111 12942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f99214a6b0fc4838bda811f10c0080c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:20:00.426546 13256 tablet_bootstrap.cc:492] T f99214a6b0fc4838bda811f10c0080c7 P 1299c2ccb7fc46bda5d4ef0a5e10033b: Bootstrap starting.
I20260809 02:20:00.427976 13257 tablet_bootstrap.cc:492] T f99214a6b0fc4838bda811f10c0080c7 P 7b2850c44df641f0a9a4b05a1049f190: Bootstrap starting.
I20260809 02:20:00.428922 13256 tablet_bootstrap.cc:654] T f99214a6b0fc4838bda811f10c0080c7 P 1299c2ccb7fc46bda5d4ef0a5e10033b: Neither blocks nor log segments found. Creating new log.
I20260809 02:20:00.429039 13258 tablet_bootstrap.cc:492] T f99214a6b0fc4838bda811f10c0080c7 P c8ef9dacda574f9782106ea90e7986eb: Bootstrap starting.
I20260809 02:20:00.429991 13257 tablet_bootstrap.cc:654] T f99214a6b0fc4838bda811f10c0080c7 P 7b2850c44df641f0a9a4b05a1049f190: Neither blocks nor log segments found. Creating new log.
I20260809 02:20:00.430295 13256 log.cc:824] T f99214a6b0fc4838bda811f10c0080c7 P 1299c2ccb7fc46bda5d4ef0a5e10033b: Log is configured to *not* fsync() on all Append() calls
I20260809 02:20:00.430898 13257 log.cc:824] T f99214a6b0fc4838bda811f10c0080c7 P 7b2850c44df641f0a9a4b05a1049f190: Log is configured to *not* fsync() on all Append() calls
I20260809 02:20:00.431128 13258 tablet_bootstrap.cc:654] T f99214a6b0fc4838bda811f10c0080c7 P c8ef9dacda574f9782106ea90e7986eb: Neither blocks nor log segments found. Creating new log.
I20260809 02:20:00.432163 13256 tablet_bootstrap.cc:492] T f99214a6b0fc4838bda811f10c0080c7 P 1299c2ccb7fc46bda5d4ef0a5e10033b: No bootstrap required, opened a new log
I20260809 02:20:00.432276 13258 log.cc:824] T f99214a6b0fc4838bda811f10c0080c7 P c8ef9dacda574f9782106ea90e7986eb: Log is configured to *not* fsync() on all Append() calls
I20260809 02:20:00.432358 13256 ts_tablet_manager.cc:1397] T f99214a6b0fc4838bda811f10c0080c7 P 1299c2ccb7fc46bda5d4ef0a5e10033b: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 02:20:00.432947 13257 tablet_bootstrap.cc:492] T f99214a6b0fc4838bda811f10c0080c7 P 7b2850c44df641f0a9a4b05a1049f190: No bootstrap required, opened a new log
I20260809 02:20:00.433141 13257 ts_tablet_manager.cc:1397] T f99214a6b0fc4838bda811f10c0080c7 P 7b2850c44df641f0a9a4b05a1049f190: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260809 02:20:00.433943 13258 tablet_bootstrap.cc:492] T f99214a6b0fc4838bda811f10c0080c7 P c8ef9dacda574f9782106ea90e7986eb: No bootstrap required, opened a new log
I20260809 02:20:00.434124 13258 ts_tablet_manager.cc:1397] T f99214a6b0fc4838bda811f10c0080c7 P c8ef9dacda574f9782106ea90e7986eb: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 02:20:00.436403 13256 raft_consensus.cc:348] T f99214a6b0fc4838bda811f10c0080c7 P 1299c2ccb7fc46bda5d4ef0a5e10033b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b2850c44df641f0a9a4b05a1049f190" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 35967 } } peers { permanent_uuid: "1299c2ccb7fc46bda5d4ef0a5e10033b" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 43159 } } peers { permanent_uuid: "c8ef9dacda574f9782106ea90e7986eb" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 36745 } }
I20260809 02:20:00.436651 13256 raft_consensus.cc:374] T f99214a6b0fc4838bda811f10c0080c7 P 1299c2ccb7fc46bda5d4ef0a5e10033b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:20:00.436753 13256 raft_consensus.cc:729] T f99214a6b0fc4838bda811f10c0080c7 P 1299c2ccb7fc46bda5d4ef0a5e10033b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1299c2ccb7fc46bda5d4ef0a5e10033b, State: Initialized, Role: FOLLOWER
I20260809 02:20:00.437227 13256 consensus_queue.cc:260] T f99214a6b0fc4838bda811f10c0080c7 P 1299c2ccb7fc46bda5d4ef0a5e10033b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b2850c44df641f0a9a4b05a1049f190" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 35967 } } peers { permanent_uuid: "1299c2ccb7fc46bda5d4ef0a5e10033b" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 43159 } } peers { permanent_uuid: "c8ef9dacda574f9782106ea90e7986eb" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 36745 } }
I20260809 02:20:00.437235 13258 raft_consensus.cc:348] T f99214a6b0fc4838bda811f10c0080c7 P c8ef9dacda574f9782106ea90e7986eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b2850c44df641f0a9a4b05a1049f190" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 35967 } } peers { permanent_uuid: "1299c2ccb7fc46bda5d4ef0a5e10033b" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 43159 } } peers { permanent_uuid: "c8ef9dacda574f9782106ea90e7986eb" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 36745 } }
I20260809 02:20:00.437492 13258 raft_consensus.cc:374] T f99214a6b0fc4838bda811f10c0080c7 P c8ef9dacda574f9782106ea90e7986eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:20:00.437425 13257 raft_consensus.cc:348] T f99214a6b0fc4838bda811f10c0080c7 P 7b2850c44df641f0a9a4b05a1049f190 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b2850c44df641f0a9a4b05a1049f190" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 35967 } } peers { permanent_uuid: "1299c2ccb7fc46bda5d4ef0a5e10033b" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 43159 } } peers { permanent_uuid: "c8ef9dacda574f9782106ea90e7986eb" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 36745 } }
I20260809 02:20:00.437630 13258 raft_consensus.cc:729] T f99214a6b0fc4838bda811f10c0080c7 P c8ef9dacda574f9782106ea90e7986eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8ef9dacda574f9782106ea90e7986eb, State: Initialized, Role: FOLLOWER
I20260809 02:20:00.437738 13257 raft_consensus.cc:374] T f99214a6b0fc4838bda811f10c0080c7 P 7b2850c44df641f0a9a4b05a1049f190 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:20:00.437820 13257 raft_consensus.cc:729] T f99214a6b0fc4838bda811f10c0080c7 P 7b2850c44df641f0a9a4b05a1049f190 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b2850c44df641f0a9a4b05a1049f190, State: Initialized, Role: FOLLOWER
I20260809 02:20:00.437944 13230 heartbeater.cc:503] Master 127.15.75.190:42661 was elected leader, sending a full tablet report...
I20260809 02:20:00.438235 13256 ts_tablet_manager.cc:1428] T f99214a6b0fc4838bda811f10c0080c7 P 1299c2ccb7fc46bda5d4ef0a5e10033b: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260809 02:20:00.438197 13258 consensus_queue.cc:260] T f99214a6b0fc4838bda811f10c0080c7 P c8ef9dacda574f9782106ea90e7986eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b2850c44df641f0a9a4b05a1049f190" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 35967 } } peers { permanent_uuid: "1299c2ccb7fc46bda5d4ef0a5e10033b" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 43159 } } peers { permanent_uuid: "c8ef9dacda574f9782106ea90e7986eb" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 36745 } }
I20260809 02:20:00.438244 13257 consensus_queue.cc:260] T f99214a6b0fc4838bda811f10c0080c7 P 7b2850c44df641f0a9a4b05a1049f190 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b2850c44df641f0a9a4b05a1049f190" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 35967 } } peers { permanent_uuid: "1299c2ccb7fc46bda5d4ef0a5e10033b" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 43159 } } peers { permanent_uuid: "c8ef9dacda574f9782106ea90e7986eb" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 36745 } }
I20260809 02:20:00.438862 12842 heartbeater.cc:503] Master 127.15.75.190:42661 was elected leader, sending a full tablet report...
I20260809 02:20:00.439100 13258 ts_tablet_manager.cc:1428] T f99214a6b0fc4838bda811f10c0080c7 P c8ef9dacda574f9782106ea90e7986eb: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260809 02:20:00.439311 13257 ts_tablet_manager.cc:1428] T f99214a6b0fc4838bda811f10c0080c7 P 7b2850c44df641f0a9a4b05a1049f190: Time spent starting tablet: real 0.006s user 0.002s sys 0.004s
I20260809 02:20:00.439442 13041 heartbeater.cc:503] Master 127.15.75.190:42661 was elected leader, sending a full tablet report...
I20260809 02:20:00.504592 13264 raft_consensus.cc:482] T f99214a6b0fc4838bda811f10c0080c7 P 1299c2ccb7fc46bda5d4ef0a5e10033b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:20:00.504890 13264 raft_consensus.cc:504] T f99214a6b0fc4838bda811f10c0080c7 P 1299c2ccb7fc46bda5d4ef0a5e10033b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b2850c44df641f0a9a4b05a1049f190" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 35967 } } peers { permanent_uuid: "1299c2ccb7fc46bda5d4ef0a5e10033b" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 43159 } } peers { permanent_uuid: "c8ef9dacda574f9782106ea90e7986eb" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 36745 } }
I20260809 02:20:00.506085 13264 leader_election.cc:288] T f99214a6b0fc4838bda811f10c0080c7 P 1299c2ccb7fc46bda5d4ef0a5e10033b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7b2850c44df641f0a9a4b05a1049f190 (127.15.75.129:35967), c8ef9dacda574f9782106ea90e7986eb (127.15.75.130:36745)
I20260809 02:20:00.510102 13252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:00.506439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.131:34785 (local address 127.15.75.129:35967)
0809 02:20:00.506573 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:00.506579 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:00.506600 (+ 21us) server_negotiation.cc:407] Connection header received
0809 02:20:00.506663 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:00.506665 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:00.506718 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:00.506824 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:20:00.507220 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:00.507751 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:00.509117 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:00.509531 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:00.509661 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:00.509716 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:00.509918 (+ 202us) server_negotiation.cc:299] Negotiation successful
0809 02:20:00.509987 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 02:20:00.510102 13233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:00.506277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.129:35967 (local address 127.15.75.131:34785)
0809 02:20:00.506403 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0809 02:20:00.506420 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 02:20:00.506517 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:20:00.506934 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:20:00.506938 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:20:00.506960 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:20:00.507075 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:20:00.507081 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:20:00.507866 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:20:00.507872 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:20:00.508984 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:20:00.508994 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:20:00.509661 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:20:00.509664 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:20:00.509779 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:00.509809 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 02:20:00.509897 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 02:20:00.509970 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260809 02:20:00.510510 13269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:00.506568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.130:36745 (local address 127.15.75.131:44283)
0809 02:20:00.506921 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0809 02:20:00.506933 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:20:00.507058 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:20:00.507327 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:20:00.507330 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:20:00.507347 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:20:00.507495 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:20:00.507501 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:20:00.508407 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:20:00.508414 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:20:00.509351 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:20:00.509361 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:20:00.510036 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:20:00.510040 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:20:00.510144 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:00.510175 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 02:20:00.510308 (+ 133us) client_negotiation.cc:240] Negotiation successful
0809 02:20:00.510375 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":180,"threads_started":1}
I20260809 02:20:00.510562 13255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:00.506705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.131:44283 (local address 127.15.75.130:36745)
0809 02:20:00.506855 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:00.506859 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:00.507023 (+ 164us) server_negotiation.cc:407] Connection header received
0809 02:20:00.507175 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:00.507181 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:00.507233 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:00.507319 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:20:00.507613 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:00.508298 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:00.509488 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:00.509910 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:00.510052 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:00.510104 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:00.510366 (+ 262us) server_negotiation.cc:299] Negotiation successful
0809 02:20:00.510446 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 02:20:00.510854 12773 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f99214a6b0fc4838bda811f10c0080c7" candidate_uuid: "1299c2ccb7fc46bda5d4ef0a5e10033b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b2850c44df641f0a9a4b05a1049f190" is_pre_election: true
I20260809 02:20:00.511191 12773 raft_consensus.cc:2454] T f99214a6b0fc4838bda811f10c0080c7 P 7b2850c44df641f0a9a4b05a1049f190 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1299c2ccb7fc46bda5d4ef0a5e10033b in term 0.
I20260809 02:20:00.511350 12971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f99214a6b0fc4838bda811f10c0080c7" candidate_uuid: "1299c2ccb7fc46bda5d4ef0a5e10033b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8ef9dacda574f9782106ea90e7986eb" is_pre_election: true
I20260809 02:20:00.511644 12971 raft_consensus.cc:2454] T f99214a6b0fc4838bda811f10c0080c7 P c8ef9dacda574f9782106ea90e7986eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1299c2ccb7fc46bda5d4ef0a5e10033b in term 0.
I20260809 02:20:00.511735 13075 leader_election.cc:302] T f99214a6b0fc4838bda811f10c0080c7 P 1299c2ccb7fc46bda5d4ef0a5e10033b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1299c2ccb7fc46bda5d4ef0a5e10033b, 7b2850c44df641f0a9a4b05a1049f190; no voters:
I20260809 02:20:00.511997 13264 raft_consensus.cc:2781] T f99214a6b0fc4838bda811f10c0080c7 P 1299c2ccb7fc46bda5d4ef0a5e10033b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:20:00.512097 13264 raft_consensus.cc:482] T f99214a6b0fc4838bda811f10c0080c7 P 1299c2ccb7fc46bda5d4ef0a5e10033b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:20:00.512159 13264 raft_consensus.cc:3037] T f99214a6b0fc4838bda811f10c0080c7 P 1299c2ccb7fc46bda5d4ef0a5e10033b [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:20:00.513415 13264 raft_consensus.cc:504] T f99214a6b0fc4838bda811f10c0080c7 P 1299c2ccb7fc46bda5d4ef0a5e10033b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b2850c44df641f0a9a4b05a1049f190" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 35967 } } peers { permanent_uuid: "1299c2ccb7fc46bda5d4ef0a5e10033b" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 43159 } } peers { permanent_uuid: "c8ef9dacda574f9782106ea90e7986eb" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 36745 } }
I20260809 02:20:00.513747 13264 leader_election.cc:288] T f99214a6b0fc4838bda811f10c0080c7 P 1299c2ccb7fc46bda5d4ef0a5e10033b [CANDIDATE]: Term 1 election: Requested vote from peers 7b2850c44df641f0a9a4b05a1049f190 (127.15.75.129:35967), c8ef9dacda574f9782106ea90e7986eb (127.15.75.130:36745)
I20260809 02:20:00.514140 12773 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f99214a6b0fc4838bda811f10c0080c7" candidate_uuid: "1299c2ccb7fc46bda5d4ef0a5e10033b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b2850c44df641f0a9a4b05a1049f190"
I20260809 02:20:00.514174 12971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f99214a6b0fc4838bda811f10c0080c7" candidate_uuid: "1299c2ccb7fc46bda5d4ef0a5e10033b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8ef9dacda574f9782106ea90e7986eb"
I20260809 02:20:00.514273 12773 raft_consensus.cc:3037] T f99214a6b0fc4838bda811f10c0080c7 P 7b2850c44df641f0a9a4b05a1049f190 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:20:00.514294 12971 raft_consensus.cc:3037] T f99214a6b0fc4838bda811f10c0080c7 P c8ef9dacda574f9782106ea90e7986eb [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:20:00.515321 12773 raft_consensus.cc:2454] T f99214a6b0fc4838bda811f10c0080c7 P 7b2850c44df641f0a9a4b05a1049f190 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1299c2ccb7fc46bda5d4ef0a5e10033b in term 1.
I20260809 02:20:00.515388 12971 raft_consensus.cc:2454] T f99214a6b0fc4838bda811f10c0080c7 P c8ef9dacda574f9782106ea90e7986eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1299c2ccb7fc46bda5d4ef0a5e10033b in term 1.
I20260809 02:20:00.515650 13075 leader_election.cc:302] T f99214a6b0fc4838bda811f10c0080c7 P 1299c2ccb7fc46bda5d4ef0a5e10033b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1299c2ccb7fc46bda5d4ef0a5e10033b, 7b2850c44df641f0a9a4b05a1049f190; no voters:
I20260809 02:20:00.515831 13264 raft_consensus.cc:2781] T f99214a6b0fc4838bda811f10c0080c7 P 1299c2ccb7fc46bda5d4ef0a5e10033b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:20:00.516084 13264 raft_consensus.cc:686] T f99214a6b0fc4838bda811f10c0080c7 P 1299c2ccb7fc46bda5d4ef0a5e10033b [term 1 LEADER]: Becoming Leader. State: Replica: 1299c2ccb7fc46bda5d4ef0a5e10033b, State: Running, Role: LEADER
I20260809 02:20:00.516429 13264 consensus_queue.cc:237] T f99214a6b0fc4838bda811f10c0080c7 P 1299c2ccb7fc46bda5d4ef0a5e10033b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b2850c44df641f0a9a4b05a1049f190" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 35967 } } peers { permanent_uuid: "1299c2ccb7fc46bda5d4ef0a5e10033b" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 43159 } } peers { permanent_uuid: "c8ef9dacda574f9782106ea90e7986eb" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 36745 } }
I20260809 02:20:00.519682 12556 catalog_manager.cc:5495] T f99214a6b0fc4838bda811f10c0080c7 P 1299c2ccb7fc46bda5d4ef0a5e10033b reported cstate change: term changed from 0 to 1, leader changed from <none> to 1299c2ccb7fc46bda5d4ef0a5e10033b (127.15.75.131). New cstate: current_term: 1 leader_uuid: "1299c2ccb7fc46bda5d4ef0a5e10033b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b2850c44df641f0a9a4b05a1049f190" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 35967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1299c2ccb7fc46bda5d4ef0a5e10033b" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 43159 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c8ef9dacda574f9782106ea90e7986eb" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 36745 } health_report { overall_health: UNKNOWN } } }
I20260809 02:20:00.540630 13254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:00.537107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34352 (local address 127.15.75.131:43159)
0809 02:20:00.537244 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:00.537248 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:00.537262 (+ 14us) server_negotiation.cc:407] Connection header received
0809 02:20:00.537401 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:00.537403 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:00.537460 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:00.537555 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:20:00.537837 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:00.538334 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:00.539214 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:00.539429 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:00.539519 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:00.539784 (+ 265us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:20:00.540232 (+ 448us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:20:00.540328 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:00.540440 (+ 112us) server_negotiation.cc:299] Negotiation successful
0809 02:20:00.540494 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 02:20:00.547137 12971 raft_consensus.cc:1261] T f99214a6b0fc4838bda811f10c0080c7 P c8ef9dacda574f9782106ea90e7986eb [term 1 FOLLOWER]: Refusing update from remote peer 1299c2ccb7fc46bda5d4ef0a5e10033b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:20:00.547163 12773 raft_consensus.cc:1261] T f99214a6b0fc4838bda811f10c0080c7 P 7b2850c44df641f0a9a4b05a1049f190 [term 1 FOLLOWER]: Refusing update from remote peer 1299c2ccb7fc46bda5d4ef0a5e10033b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:20:00.547891 13264 consensus_queue.cc:1035] T f99214a6b0fc4838bda811f10c0080c7 P 1299c2ccb7fc46bda5d4ef0a5e10033b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b2850c44df641f0a9a4b05a1049f190" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 35967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:20:00.548107 13271 consensus_queue.cc:1035] T f99214a6b0fc4838bda811f10c0080c7 P 1299c2ccb7fc46bda5d4ef0a5e10033b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8ef9dacda574f9782106ea90e7986eb" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 36745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:20:00.554185 13275 mvcc.cc:205] Tried to move back new op lower bound from 7316447234233634816 to 7316447234118098944. Current Snapshot: MvccSnapshot[applied={T|T < 7316447234233634816}]
I20260809 02:20:00.557266 13277 mvcc.cc:205] Tried to move back new op lower bound from 7316447234233634816 to 7316447234118098944. Current Snapshot: MvccSnapshot[applied={T|T < 7316447234233634816}]
I20260809 02:20:00.557802 13276 mvcc.cc:205] Tried to move back new op lower bound from 7316447234233634816 to 7316447234118098944. Current Snapshot: MvccSnapshot[applied={T|T < 7316447234233634816}]
I20260809 02:20:00.563709 12556 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53272:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:20:00.563882 12556 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53272:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:20:00.566330 12556 catalog_manager.cc:5782] T 00000000000000000000000000000000 P df9aab2447d044a1a930354ef70ecc03: Sending DeleteTablet for 3 replicas of tablet f99214a6b0fc4838bda811f10c0080c7
I20260809 02:20:00.567257 13135 tablet_service.cc:1514] Processing DeleteTablet for tablet f99214a6b0fc4838bda811f10c0080c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:20:00 UTC) from {username='kudu', principal='kudu/127.15.75.190@KRBTEST.COM'} at 127.0.0.1:34350
I20260809 02:20:00.567255 12747 tablet_service.cc:1514] Processing DeleteTablet for tablet f99214a6b0fc4838bda811f10c0080c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:20:00 UTC) from {username='kudu', principal='kudu/127.15.75.190@KRBTEST.COM'} at 127.0.0.1:45140
I20260809 02:20:00.567281 12942 tablet_service.cc:1514] Processing DeleteTablet for tablet f99214a6b0fc4838bda811f10c0080c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:20:00 UTC) from {username='kudu', principal='kudu/127.15.75.190@KRBTEST.COM'} at 127.0.0.1:41694
I20260809 02:20:00.567770 13288 tablet_replica.cc:330] T f99214a6b0fc4838bda811f10c0080c7 P 1299c2ccb7fc46bda5d4ef0a5e10033b: stopping tablet replica
I20260809 02:20:00.567798 13291 tablet_replica.cc:330] T f99214a6b0fc4838bda811f10c0080c7 P c8ef9dacda574f9782106ea90e7986eb: stopping tablet replica
I20260809 02:20:00.568063 13289 tablet_replica.cc:330] T f99214a6b0fc4838bda811f10c0080c7 P 7b2850c44df641f0a9a4b05a1049f190: stopping tablet replica
I20260809 02:20:00.568434 13289 raft_consensus.cc:2229] T f99214a6b0fc4838bda811f10c0080c7 P 7b2850c44df641f0a9a4b05a1049f190 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:20:00.568460 13291 raft_consensus.cc:2229] T f99214a6b0fc4838bda811f10c0080c7 P c8ef9dacda574f9782106ea90e7986eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:20:00.568560 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 12642
I20260809 02:20:00.568783 13291 raft_consensus.cc:2258] T f99214a6b0fc4838bda811f10c0080c7 P c8ef9dacda574f9782106ea90e7986eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:20:00.568783 13289 raft_consensus.cc:2258] T f99214a6b0fc4838bda811f10c0080c7 P 7b2850c44df641f0a9a4b05a1049f190 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:20:00.572438 13291 ts_tablet_manager.cc:1905] T f99214a6b0fc4838bda811f10c0080c7 P c8ef9dacda574f9782106ea90e7986eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:20:00.575999 13288 raft_consensus.cc:2229] T f99214a6b0fc4838bda811f10c0080c7 P 1299c2ccb7fc46bda5d4ef0a5e10033b [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:20:00.576191 13291 ts_tablet_manager.cc:1918] T f99214a6b0fc4838bda811f10c0080c7 P c8ef9dacda574f9782106ea90e7986eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:20:00.576296 13291 log.cc:1196] T f99214a6b0fc4838bda811f10c0080c7 P c8ef9dacda574f9782106ea90e7986eb: Deleting WAL directory at /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/minicluster-data/ts-1/wal/wals/f99214a6b0fc4838bda811f10c0080c7
I20260809 02:20:00.576642 13291 ts_tablet_manager.cc:1939] T f99214a6b0fc4838bda811f10c0080c7 P c8ef9dacda574f9782106ea90e7986eb: Deleting consensus metadata
I20260809 02:20:00.576776 13288 raft_consensus.cc:2258] T f99214a6b0fc4838bda811f10c0080c7 P 1299c2ccb7fc46bda5d4ef0a5e10033b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:20:00.578235 12521 catalog_manager.cc:4841] TS c8ef9dacda574f9782106ea90e7986eb (127.15.75.130:36745): tablet f99214a6b0fc4838bda811f10c0080c7 (table test-table [id=7a58484fa14d480d94ebfccfa131a21b]) successfully deleted
W20260809 02:20:00.583806 12521 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.75.129:35967 (error 108)
I20260809 02:20:00.583982 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 12847
I20260809 02:20:00.586884 13250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:00.586543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.129:35967 (local address 127.0.0.1:45152)
0809 02:20:00.586696 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 02:20:00.586779 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.129:35967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260809 02:20:00.588587 12521 catalog_manager.cc:4584] TS 7b2850c44df641f0a9a4b05a1049f190 (127.15.75.129:35967): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f99214a6b0fc4838bda811f10c0080c7: Network error: Client connection negotiation failed: client connection to 127.15.75.129:35967: connect: Connection refused (error 111)
I20260809 02:20:00.590626 13288 ts_tablet_manager.cc:1905] T f99214a6b0fc4838bda811f10c0080c7 P 1299c2ccb7fc46bda5d4ef0a5e10033b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:20:00.593377 13288 ts_tablet_manager.cc:1918] T f99214a6b0fc4838bda811f10c0080c7 P 1299c2ccb7fc46bda5d4ef0a5e10033b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:20:00.593482 13288 log.cc:1196] T f99214a6b0fc4838bda811f10c0080c7 P 1299c2ccb7fc46bda5d4ef0a5e10033b: Deleting WAL directory at /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241741788589-15662-0/minicluster-data/ts-2/wal/wals/f99214a6b0fc4838bda811f10c0080c7
I20260809 02:20:00.593827 13288 ts_tablet_manager.cc:1939] T f99214a6b0fc4838bda811f10c0080c7 P 1299c2ccb7fc46bda5d4ef0a5e10033b: Deleting consensus metadata
I20260809 02:20:00.595145 12520 catalog_manager.cc:4841] TS 1299c2ccb7fc46bda5d4ef0a5e10033b (127.15.75.131:43159): tablet f99214a6b0fc4838bda811f10c0080c7 (table test-table [id=7a58484fa14d480d94ebfccfa131a21b]) successfully deleted
I20260809 02:20:00.595211 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 13049
I20260809 02:20:00.604489 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 12498
2026-08-09T02:20:00Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2509 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-09T02:20:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:20:00Z Disabled control of system clock
I20260809 02:20:00.639710 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241741788589-15662-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241741788589-15662-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241741788589-15662-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:42471
--webserver_interface=127.15.75.190
--webserver_port=0
--builtin_ntp_servers=127.15.75.148:43689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.75.190:42471
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 02:20:00.750344 13299 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:20:00.750566 13299 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:20:00.750600 13299 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:20:00.754403 13299 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:20:00.754444 13299 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:20:00.754459 13299 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:20:00.754475 13299 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:20:00.754490 13299 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:20:00.759068 13299 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:43689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241741788589-15662-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241741788589-15662-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.75.190:42471
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:42471
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.75.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13299
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241741788589-15662-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:20:00.759940 13299 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:20:00.761162 13299 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:20:00.769227 13313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:20:00.769239 13306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:20:00.769241 13307 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:20:00.769286 13299 server_base.cc:1034] running on GCE node
I20260809 02:20:00.769987 13299 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:20:00.771045 13299 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:20:00.772228 13299 hybrid_clock.cc:648] HybridClock initialized: now 1786242000772198 us; error 44 us; skew 500 ppm
I20260809 02:20:00.774024 13299 webserver.cc:460] Webserver started at http://127.15.75.190:36863/ using document root <none> and password file <none>
I20260809 02:20:00.774595 13299 fs_manager.cc:359] Metadata directory not provided
I20260809 02:20:00.774686 13299 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:20:00.774933 13299 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:20:00.776985 13299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241741788589-15662-0/minicluster-data/master-0/data/instance:
uuid: "86196f16428e4df0a5a3072eb29b1ecb"
format_stamp: "Formatted at 2026-08-09 02:20:00 on dist-test-slave-s11t"
server_key: "379c04019a4f1e808540919ed959f39f"
server_key_iv: "33e3274ccd61d21277de7f275657ccc5"
server_key_version: "encryptionkey@0"
I20260809 02:20:00.777530 13299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241741788589-15662-0/minicluster-data/master-0/wal/instance:
uuid: "86196f16428e4df0a5a3072eb29b1ecb"
format_stamp: "Formatted at 2026-08-09 02:20:00 on dist-test-slave-s11t"
server_key: "379c04019a4f1e808540919ed959f39f"
server_key_iv: "33e3274ccd61d21277de7f275657ccc5"
server_key_version: "encryptionkey@0"
I20260809 02:20:00.780964 13299 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 02:20:00.783270 13322 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:20:00.784489 13299 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 02:20:00.784626 13299 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241741788589-15662-0/minicluster-data/master-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241741788589-15662-0/minicluster-data/master-0/wal
uuid: "86196f16428e4df0a5a3072eb29b1ecb"
format_stamp: "Formatted at 2026-08-09 02:20:00 on dist-test-slave-s11t"
server_key: "379c04019a4f1e808540919ed959f39f"
server_key_iv: "33e3274ccd61d21277de7f275657ccc5"
server_key_version: "encryptionkey@0"
I20260809 02:20:00.784777 13299 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241741788589-15662-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241741788589-15662-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241741788589-15662-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:20:00.797874 13299 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:20:00.798456 13299 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:20:00.798641 13299 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:20:00.805547 13299 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.190:42471
I20260809 02:20:00.805568 13392 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.190:42471 every 8 connection(s)
I20260809 02:20:00.806445 13299 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
I20260809 02:20:00.809486 13393 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:20:00.814791 13393 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86196f16428e4df0a5a3072eb29b1ecb: Bootstrap starting.
I20260809 02:20:00.815637 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 13299
I20260809 02:20:00.815919 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241741788589-15662-0/minicluster-data/master-0/wal/instance
I20260809 02:20:00.816349 15662 external_mini_cluster.cc:1428] Setting key 1db62e2bb06534aaaf6abbb4f373d9b5
I20260809 02:20:00.817564 13393 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 86196f16428e4df0a5a3072eb29b1ecb: Neither blocks nor log segments found. Creating new log.
I20260809 02:20:00.818367 13393 log.cc:824] T 00000000000000000000000000000000 P 86196f16428e4df0a5a3072eb29b1ecb: Log is configured to *not* fsync() on all Append() calls
I20260809 02:20:00.820444 13393 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86196f16428e4df0a5a3072eb29b1ecb: No bootstrap required, opened a new log
I20260809 02:20:00.824050 13393 raft_consensus.cc:348] T 00000000000000000000000000000000 P 86196f16428e4df0a5a3072eb29b1ecb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86196f16428e4df0a5a3072eb29b1ecb" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 42471 } }
I20260809 02:20:00.824292 13393 raft_consensus.cc:374] T 00000000000000000000000000000000 P 86196f16428e4df0a5a3072eb29b1ecb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:20:00.824360 13393 raft_consensus.cc:729] T 00000000000000000000000000000000 P 86196f16428e4df0a5a3072eb29b1ecb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86196f16428e4df0a5a3072eb29b1ecb, State: Initialized, Role: FOLLOWER
I20260809 02:20:00.824862 13393 consensus_queue.cc:260] T 00000000000000000000000000000000 P 86196f16428e4df0a5a3072eb29b1ecb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86196f16428e4df0a5a3072eb29b1ecb" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 42471 } }
I20260809 02:20:00.825016 13393 raft_consensus.cc:388] T 00000000000000000000000000000000 P 86196f16428e4df0a5a3072eb29b1ecb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:20:00.825103 13393 raft_consensus.cc:482] T 00000000000000000000000000000000 P 86196f16428e4df0a5a3072eb29b1ecb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:20:00.825213 13393 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 86196f16428e4df0a5a3072eb29b1ecb [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:20:00.825742 13396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:00.817522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49868 (local address 127.15.75.190:42471)
0809 02:20:00.818143 (+ 621us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:00.818156 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:00.818189 (+ 33us) server_negotiation.cc:407] Connection header received
0809 02:20:00.818928 (+ 739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:00.818953 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:00.819374 (+ 421us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:00.819697 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:00.820107 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:00.821197 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:00.822122 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:00.822594 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:00.822789 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:00.823713 (+ 924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:20:00.823719 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:20:00.823731 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:20:00.824101 (+ 370us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:20:00.824155 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:20:00.824174 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:20:00.824304 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:00.824453 (+ 149us) server_negotiation.cc:299] Negotiation successful
0809 02:20:00.824732 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":128,"threads_started":1}
I20260809 02:20:00.827044 13393 raft_consensus.cc:504] T 00000000000000000000000000000000 P 86196f16428e4df0a5a3072eb29b1ecb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86196f16428e4df0a5a3072eb29b1ecb" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 42471 } }
I20260809 02:20:00.827394 13393 leader_election.cc:302] T 00000000000000000000000000000000 P 86196f16428e4df0a5a3072eb29b1ecb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 86196f16428e4df0a5a3072eb29b1ecb; no voters:
I20260809 02:20:00.827723 13393 leader_election.cc:288] T 00000000000000000000000000000000 P 86196f16428e4df0a5a3072eb29b1ecb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:20:00.827826 13398 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 86196f16428e4df0a5a3072eb29b1ecb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:20:00.828049 13398 raft_consensus.cc:686] T 00000000000000000000000000000000 P 86196f16428e4df0a5a3072eb29b1ecb [term 1 LEADER]: Becoming Leader. State: Replica: 86196f16428e4df0a5a3072eb29b1ecb, State: Running, Role: LEADER
I20260809 02:20:00.828338 13398 consensus_queue.cc:237] T 00000000000000000000000000000000 P 86196f16428e4df0a5a3072eb29b1ecb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86196f16428e4df0a5a3072eb29b1ecb" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 42471 } }
I20260809 02:20:00.828967 13393 sys_catalog.cc:564] T 00000000000000000000000000000000 P 86196f16428e4df0a5a3072eb29b1ecb [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:20:00.830084 13399 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86196f16428e4df0a5a3072eb29b1ecb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "86196f16428e4df0a5a3072eb29b1ecb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86196f16428e4df0a5a3072eb29b1ecb" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 42471 } } }
I20260809 02:20:00.830230 13399 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86196f16428e4df0a5a3072eb29b1ecb [sys.catalog]: This master's current role is: LEADER
I20260809 02:20:00.830641 13407 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:20:00.830546 13400 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86196f16428e4df0a5a3072eb29b1ecb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 86196f16428e4df0a5a3072eb29b1ecb. Latest consensus state: current_term: 1 leader_uuid: "86196f16428e4df0a5a3072eb29b1ecb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86196f16428e4df0a5a3072eb29b1ecb" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 42471 } } }
I20260809 02:20:00.831229 13400 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86196f16428e4df0a5a3072eb29b1ecb [sys.catalog]: This master's current role is: LEADER
I20260809 02:20:00.833715 13407 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:20:00.839360 13407 catalog_manager.cc:1347] Generated new cluster ID: a11d3a7946534801b2427561d208f4ee
I20260809 02:20:00.839430 13407 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:20:00.851317 13407 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:20:00.852185 13407 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:20:00.861001 13407 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 86196f16428e4df0a5a3072eb29b1ecb: Generated new TSK 0
I20260809 02:20:00.861663 13407 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:20:00.864264 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241741788589-15662-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241741788589-15662-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241741788589-15662-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.129:0
--local_ip_for_outbound_sockets=127.15.75.129
--webserver_interface=127.15.75.129
--webserver_port=0
--tserver_master_addrs=127.15.75.190:42471
--builtin_ntp_servers=127.15.75.148:43689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 02:20:00.992256 13424 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:20:00.992507 13424 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:20:00.992547 13424 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:20:00.996491 13424 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:20:00.996537 13424 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:20:00.996626 13424 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.129
I20260809 02:20:01.001293 13424 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:43689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241741788589-15662-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241741788589-15662-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.75.129
--webserver_port=0
--tserver_master_addrs=127.15.75.190:42471
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13424
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.129
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241741788589-15662-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:20:01.002255 13424 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:20:01.003870 13424 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 02:20:01.013583 13424 server_base.cc:1034] running on GCE node
W20260809 02:20:01.013549 13430 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:20:01.013538 13434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:20:01.013537 13429 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:20:01.014315 13424 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:20:01.014909 13424 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:20:01.016098 13424 hybrid_clock.cc:648] HybridClock initialized: now 1786242001016065 us; error 39 us; skew 500 ppm
I20260809 02:20:01.018098 13424 webserver.cc:460] Webserver started at http://127.15.75.129:39661/ using document root <none> and password file <none>
I20260809 02:20:01.018680 13424 fs_manager.cc:359] Metadata directory not provided
I20260809 02:20:01.018770 13424 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:20:01.019008 13424 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:20:01.021091 13424 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241741788589-15662-0/minicluster-data/ts-0/data/instance:
uuid: "24c4ebc315d1493bb5c6c68a480e8750"
format_stamp: "Formatted at 2026-08-09 02:20:01 on dist-test-slave-s11t"
server_key: "a43596c7165c3895a92441f08b77f040"
server_key_iv: "ec2cdfe6365f88265ce27cdad92a0a31"
server_key_version: "encryptionkey@0"
I20260809 02:20:01.021682 13424 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241741788589-15662-0/minicluster-data/ts-0/wal/instance:
uuid: "24c4ebc315d1493bb5c6c68a480e8750"
format_stamp: "Formatted at 2026-08-09 02:20:01 on dist-test-slave-s11t"
server_key: "a43596c7165c3895a92441f08b77f040"
server_key_iv: "ec2cdfe6365f88265ce27cdad92a0a31"
server_key_version: "encryptionkey@0"
I20260809 02:20:01.025259 13424 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 02:20:01.027565 13448 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:20:01.028725 13424 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 02:20:01.028867 13424 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241741788589-15662-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241741788589-15662-0/minicluster-data/ts-0/wal
uuid: "24c4ebc315d1493bb5c6c68a480e8750"
format_stamp: "Formatted at 2026-08-09 02:20:01 on dist-test-slave-s11t"
server_key: "a43596c7165c3895a92441f08b77f040"
server_key_iv: "ec2cdfe6365f88265ce27cdad92a0a31"
server_key_version: "encryptionkey@0"
I20260809 02:20:01.028967 13424 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241741788589-15662-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241741788589-15662-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241741788589-15662-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:20:01.057478 13424 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:20:01.058176 13424 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:20:01.058385 13424 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:20:01.058979 13424 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:20:01.059909 13424 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:20:01.059957 13424 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:20:01.060027 13424 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:20:01.060058 13424 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:20:01.069422 13424 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.129:33673
I20260809 02:20:01.069495 13605 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.129:33673 every 8 connection(s)
I20260809 02:20:01.070381 13424 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241741788589-15662-0/minicluster-data/ts-0/data/info.pb
I20260809 02:20:01.072278 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 13424
I20260809 02:20:01.072412 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241741788589-15662-0/minicluster-data/ts-0/wal/instance
I20260809 02:20:01.072777 15662 external_mini_cluster.cc:1428] Setting key 8e1fbced3c7612bf830e6bdaa15dda6a
I20260809 02:20:01.074916 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241741788589-15662-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241741788589-15662-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241741788589-15662-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.130:0
--local_ip_for_outbound_sockets=127.15.75.130
--webserver_interface=127.15.75.130
--webserver_port=0
--tserver_master_addrs=127.15.75.190:42471
--builtin_ntp_servers=127.15.75.148:43689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 02:20:01.080161 13396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:01.072178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.129:34525 (local address 127.15.75.190:42471)
0809 02:20:01.072329 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:01.072350 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:01.073696 (+ 1346us) server_negotiation.cc:407] Connection header received
0809 02:20:01.075415 (+ 1719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:01.075418 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:01.075473 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:01.075550 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:01.076413 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:01.077036 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:01.078176 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:01.078407 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:01.078517 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:01.079675 (+ 1158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:20:01.079683 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:20:01.079686 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:20:01.079697 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:20:01.079717 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:20:01.079725 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:20:01.079804 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:01.079962 (+ 158us) server_negotiation.cc:299] Negotiation successful
0809 02:20:01.080022 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 02:20:01.081040 13614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:01.072712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:42471 (local address 127.15.75.129:34525)
0809 02:20:01.073532 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0809 02:20:01.073573 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 02:20:01.075199 (+ 1626us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:20:01.075731 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:20:01.075738 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:20:01.075820 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:20:01.076276 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:20:01.076288 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:20:01.077167 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:20:01.077171 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:20:01.078046 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:20:01.078055 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:20:01.078568 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:20:01.078573 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:20:01.078765 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:01.079423 (+ 658us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:20:01.079425 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:20:01.079463 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:20:01.079522 (+ 59us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:20:01.079807 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:20:01.079819 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:20:01.079829 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 02:20:01.079961 (+ 132us) client_negotiation.cc:240] Negotiation successful
0809 02:20:01.080222 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":165,"threads_started":1}
I20260809 02:20:01.082312 13608 heartbeater.cc:347] Connected to a master server at 127.15.75.190:42471
I20260809 02:20:01.082556 13608 heartbeater.cc:464] Registering TS with master...
I20260809 02:20:01.083083 13608 heartbeater.cc:511] Master 127.15.75.190:42471 requested a full tablet report, sending...
I20260809 02:20:01.084779 13345 ts_manager.cc:194] Registered new tserver with Master: 24c4ebc315d1493bb5c6c68a480e8750 (127.15.75.129:33673)
I20260809 02:20:01.086516 13345 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.75.129:34525
W20260809 02:20:01.186906 13615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:20:01.187131 13615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:20:01.187167 13615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:20:01.192233 13615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:20:01.192298 13615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:20:01.192416 13615 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.130
I20260809 02:20:01.200335 13615 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:43689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241741788589-15662-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241741788589-15662-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.75.130
--webserver_port=0
--tserver_master_addrs=127.15.75.190:42471
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13615
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.130
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241741788589-15662-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:20:01.201722 13615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:20:01.203188 13615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:20:01.212587 13623 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:20:01.212561 13634 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:20:01.212576 13626 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:20:01.212787 13615 server_base.cc:1034] running on GCE node
I20260809 02:20:01.213243 13615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:20:01.213825 13615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:20:01.214982 13615 hybrid_clock.cc:648] HybridClock initialized: now 1786242001214968 us; error 25 us; skew 500 ppm
I20260809 02:20:01.216663 13615 webserver.cc:460] Webserver started at http://127.15.75.130:33645/ using document root <none> and password file <none>
I20260809 02:20:01.217293 13615 fs_manager.cc:359] Metadata directory not provided
I20260809 02:20:01.217355 13615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:20:01.217592 13615 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:20:01.219563 13615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241741788589-15662-0/minicluster-data/ts-1/data/instance:
uuid: "fe2cbb6e2e274da8b21d4062683e5cef"
format_stamp: "Formatted at 2026-08-09 02:20:01 on dist-test-slave-s11t"
server_key: "8d8d904c3df1db61b066c4f5436e6ec7"
server_key_iv: "dfb61ed73d8c8e9ba824346a551f884f"
server_key_version: "encryptionkey@0"
I20260809 02:20:01.220109 13615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241741788589-15662-0/minicluster-data/ts-1/wal/instance:
uuid: "fe2cbb6e2e274da8b21d4062683e5cef"
format_stamp: "Formatted at 2026-08-09 02:20:01 on dist-test-slave-s11t"
server_key: "8d8d904c3df1db61b066c4f5436e6ec7"
server_key_iv: "dfb61ed73d8c8e9ba824346a551f884f"
server_key_version: "encryptionkey@0"
I20260809 02:20:01.223681 13615 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260809 02:20:01.226037 13642 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:20:01.227125 13615 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 02:20:01.227240 13615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241741788589-15662-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241741788589-15662-0/minicluster-data/ts-1/wal
uuid: "fe2cbb6e2e274da8b21d4062683e5cef"
format_stamp: "Formatted at 2026-08-09 02:20:01 on dist-test-slave-s11t"
server_key: "8d8d904c3df1db61b066c4f5436e6ec7"
server_key_iv: "dfb61ed73d8c8e9ba824346a551f884f"
server_key_version: "encryptionkey@0"
I20260809 02:20:01.227337 13615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241741788589-15662-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241741788589-15662-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241741788589-15662-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:20:01.244005 13615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:20:01.244656 13615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:20:01.244877 13615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:20:01.245417 13615 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:20:01.246306 13615 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:20:01.246352 13615 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:20:01.246421 13615 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:20:01.246459 13615 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:20:01.255460 13615 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.130:45931
I20260809 02:20:01.255493 13816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.130:45931 every 8 connection(s)
I20260809 02:20:01.256306 13615 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241741788589-15662-0/minicluster-data/ts-1/data/info.pb
I20260809 02:20:01.261932 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 13615
I20260809 02:20:01.262068 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241741788589-15662-0/minicluster-data/ts-1/wal/instance
I20260809 02:20:01.262377 15662 external_mini_cluster.cc:1428] Setting key a7a7ba6617dbf14b9a4ceedf694444ed
I20260809 02:20:01.264289 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241741788589-15662-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241741788589-15662-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241741788589-15662-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.75.131:0
--local_ip_for_outbound_sockets=127.15.75.131
--webserver_interface=127.15.75.131
--webserver_port=0
--tserver_master_addrs=127.15.75.190:42471
--builtin_ntp_servers=127.15.75.148:43689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 02:20:01.265596 13396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:01.258082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.130:43293 (local address 127.15.75.190:42471)
0809 02:20:01.258226 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:01.258230 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:01.259219 (+ 989us) server_negotiation.cc:407] Connection header received
0809 02:20:01.260608 (+ 1389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:01.260612 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:01.260669 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:01.260793 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:01.261632 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:01.262290 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:01.263586 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:01.263815 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:01.263928 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:01.265108 (+ 1180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:20:01.265112 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:20:01.265114 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:20:01.265126 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:20:01.265145 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:20:01.265153 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:20:01.265232 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:01.265395 (+ 163us) server_negotiation.cc:299] Negotiation successful
0809 02:20:01.265458 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 02:20:01.266539 13821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:01.258441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:42471 (local address 127.15.75.130:43293)
0809 02:20:01.259071 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0809 02:20:01.259109 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 02:20:01.260400 (+ 1291us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:20:01.260969 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:20:01.260976 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:20:01.261053 (+ 77us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:20:01.261492 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:20:01.261504 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:20:01.262428 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:20:01.262431 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:20:01.263446 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:20:01.263457 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:20:01.263968 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:20:01.263972 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:20:01.264132 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:01.264834 (+ 702us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:20:01.264837 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:20:01.264892 (+ 55us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:20:01.264952 (+ 60us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:20:01.265234 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:20:01.265246 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:20:01.265255 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 02:20:01.265391 (+ 136us) client_negotiation.cc:240] Negotiation successful
0809 02:20:01.265647 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":125,"threads_started":1}
I20260809 02:20:01.267783 13817 heartbeater.cc:347] Connected to a master server at 127.15.75.190:42471
I20260809 02:20:01.268023 13817 heartbeater.cc:464] Registering TS with master...
I20260809 02:20:01.268512 13817 heartbeater.cc:511] Master 127.15.75.190:42471 requested a full tablet report, sending...
I20260809 02:20:01.269552 13345 ts_manager.cc:194] Registered new tserver with Master: fe2cbb6e2e274da8b21d4062683e5cef (127.15.75.130:45931)
I20260809 02:20:01.270115 13345 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.75.130:43293
W20260809 02:20:01.376886 13822 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:20:01.377135 13822 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:20:01.377199 13822 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:20:01.381173 13822 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:20:01.381240 13822 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:20:01.381317 13822 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.75.131
I20260809 02:20:01.385953 13822 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:43689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241741788589-15662-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241741788589-15662-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.75.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.75.131
--webserver_port=0
--tserver_master_addrs=127.15.75.190:42471
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13822
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.75.131
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241741788589-15662-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:20:01.386880 13822 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:20:01.388200 13822 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:20:01.397585 13832 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:20:01.397596 13830 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:20:01.397596 13829 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:20:01.397861 13822 server_base.cc:1034] running on GCE node
I20260809 02:20:01.398234 13822 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:20:01.398825 13822 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:20:01.400008 13822 hybrid_clock.cc:648] HybridClock initialized: now 1786242001399993 us; error 25 us; skew 500 ppm
I20260809 02:20:01.402040 13822 webserver.cc:460] Webserver started at http://127.15.75.131:44381/ using document root <none> and password file <none>
I20260809 02:20:01.402613 13822 fs_manager.cc:359] Metadata directory not provided
I20260809 02:20:01.402678 13822 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:20:01.402972 13822 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:20:01.405094 13822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241741788589-15662-0/minicluster-data/ts-2/data/instance:
uuid: "7c2f822d288d4f5b9bfe85978573d75e"
format_stamp: "Formatted at 2026-08-09 02:20:01 on dist-test-slave-s11t"
server_key: "7785cdd05f2a6f30c91300d611640e6b"
server_key_iv: "8baa77157d047176518f685ffa029b18"
server_key_version: "encryptionkey@0"
I20260809 02:20:01.405666 13822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241741788589-15662-0/minicluster-data/ts-2/wal/instance:
uuid: "7c2f822d288d4f5b9bfe85978573d75e"
format_stamp: "Formatted at 2026-08-09 02:20:01 on dist-test-slave-s11t"
server_key: "7785cdd05f2a6f30c91300d611640e6b"
server_key_iv: "8baa77157d047176518f685ffa029b18"
server_key_version: "encryptionkey@0"
I20260809 02:20:01.409350 13822 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 02:20:01.411743 13840 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:20:01.413074 13822 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 02:20:01.413198 13822 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241741788589-15662-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241741788589-15662-0/minicluster-data/ts-2/wal
uuid: "7c2f822d288d4f5b9bfe85978573d75e"
format_stamp: "Formatted at 2026-08-09 02:20:01 on dist-test-slave-s11t"
server_key: "7785cdd05f2a6f30c91300d611640e6b"
server_key_iv: "8baa77157d047176518f685ffa029b18"
server_key_version: "encryptionkey@0"
I20260809 02:20:01.413305 13822 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241741788589-15662-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241741788589-15662-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241741788589-15662-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:20:01.446112 13822 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:20:01.446813 13822 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:20:01.447019 13822 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:20:01.447603 13822 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:20:01.448556 13822 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:20:01.448630 13822 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:20:01.448696 13822 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:20:01.448805 13822 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:20:01.457715 13822 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.131:38067
I20260809 02:20:01.457801 14028 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.131:38067 every 8 connection(s)
I20260809 02:20:01.458662 13822 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241741788589-15662-0/minicluster-data/ts-2/data/info.pb
I20260809 02:20:01.462159 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 13822
I20260809 02:20:01.462302 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241741788589-15662-0/minicluster-data/ts-2/wal/instance
I20260809 02:20:01.462585 15662 external_mini_cluster.cc:1428] Setting key 5dafe7fa7500451ae3392afc3b4e2441
I20260809 02:20:01.468122 13396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:01.460535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.131:53557 (local address 127.15.75.190:42471)
0809 02:20:01.460742 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:01.460747 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:01.461793 (+ 1046us) server_negotiation.cc:407] Connection header received
0809 02:20:01.463386 (+ 1593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:01.463391 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:01.463453 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:01.463555 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:01.464548 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:01.465223 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:01.466347 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:01.466563 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:01.466643 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:01.467655 (+ 1012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:20:01.467658 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:20:01.467660 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:20:01.467672 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:20:01.467691 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:20:01.467698 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:20:01.467789 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:01.467960 (+ 171us) server_negotiation.cc:299] Negotiation successful
0809 02:20:01.468011 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 02:20:01.469048 14032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:01.460964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:42471 (local address 127.15.75.131:53557)
0809 02:20:01.461614 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0809 02:20:01.461656 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 02:20:01.463154 (+ 1498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:20:01.463740 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:20:01.463747 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:20:01.463829 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:20:01.464412 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:20:01.464426 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:20:01.465346 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:20:01.465348 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:20:01.466239 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:20:01.466248 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:20:01.466700 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:20:01.466703 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:20:01.466850 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:01.467408 (+ 558us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:20:01.467409 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:20:01.467449 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:20:01.467509 (+ 60us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:20:01.467827 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:20:01.467835 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:20:01.467840 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 02:20:01.467944 (+ 104us) client_negotiation.cc:240] Negotiation successful
0809 02:20:01.468158 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":117,"threads_started":1}
I20260809 02:20:01.470194 14029 heartbeater.cc:347] Connected to a master server at 127.15.75.190:42471
I20260809 02:20:01.470432 14029 heartbeater.cc:464] Registering TS with master...
I20260809 02:20:01.470980 14029 heartbeater.cc:511] Master 127.15.75.190:42471 requested a full tablet report, sending...
I20260809 02:20:01.472141 13345 ts_manager.cc:194] Registered new tserver with Master: 7c2f822d288d4f5b9bfe85978573d75e (127.15.75.131:38067)
I20260809 02:20:01.472832 13345 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.75.131:53557
I20260809 02:20:01.476500 15662 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 02:20:01.482517 13396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:01.478685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49872 (local address 127.15.75.190:42471)
0809 02:20:01.478854 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:01.478858 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:01.479025 (+ 167us) server_negotiation.cc:407] Connection header received
0809 02:20:01.479119 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:01.479121 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:01.479159 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:01.479236 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:01.479549 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:01.480056 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:01.480994 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:01.481214 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:01.481311 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:01.482051 (+ 740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:20:01.482054 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:20:01.482055 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:20:01.482067 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:20:01.482083 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:20:01.482091 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:20:01.482151 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:01.482264 (+ 113us) server_negotiation.cc:299] Negotiation successful
0809 02:20:01.482340 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260809 02:20:01.485193 13345 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49872:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:20:01.488169 13345 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:20:01.502501 14044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:01.496986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.129:33673 (local address 127.0.0.1:45996)
0809 02:20:01.497928 (+ 942us) negotiation.cc:107] Waiting for socket to connect
0809 02:20:01.497941 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:20:01.498035 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:20:01.498439 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:20:01.498442 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:20:01.498460 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:20:01.498570 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:20:01.498579 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:20:01.499636 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:20:01.499641 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:20:01.500989 (+ 1348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:20:01.501000 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:20:01.501943 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:20:01.501949 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:20:01.502056 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:01.502092 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 02:20:01.502207 (+ 115us) client_negotiation.cc:240] Negotiation successful
0809 02:20:01.502321 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":878,"mutex_wait_us":38,"thread_start_us":102,"threads_started":1}
I20260809 02:20:01.502535 14049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:01.497570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45996 (local address 127.15.75.129:33673)
0809 02:20:01.498055 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:01.498070 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:01.498102 (+ 32us) server_negotiation.cc:407] Connection header received
0809 02:20:01.498164 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:01.498167 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:01.498303 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:01.498457 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:20:01.498704 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:01.499514 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:01.501115 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:01.501808 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:01.501919 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:01.502104 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:01.502261 (+ 157us) server_negotiation.cc:299] Negotiation successful
0809 02:20:01.502368 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":155,"threads_started":1}
I20260809 02:20:01.502818 14043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:01.496511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.131:38067 (local address 127.0.0.1:42710)
0809 02:20:01.497301 (+ 790us) negotiation.cc:107] Waiting for socket to connect
0809 02:20:01.497348 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 02:20:01.497474 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:20:01.498258 (+ 784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:20:01.498266 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:20:01.498290 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:20:01.498421 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:20:01.498428 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:20:01.499857 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:20:01.499860 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:20:01.501172 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:20:01.501183 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:20:01.502248 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:20:01.502251 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:20:01.502344 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:01.502364 (+ 20us) client_negotiation.cc:769] Sending connection context
0809 02:20:01.502431 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 02:20:01.502483 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":624,"mutex_wait_us":32,"thread_start_us":78,"threads_started":1}
I20260809 02:20:01.502872 14045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:01.496642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42710 (local address 127.15.75.131:38067)
0809 02:20:01.497272 (+ 630us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:01.497287 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:01.497629 (+ 342us) server_negotiation.cc:407] Connection header received
0809 02:20:01.497711 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:01.497718 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:01.497939 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:01.498087 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:20:01.498553 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:01.499718 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:01.501306 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:01.501907 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:01.502011 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:01.502187 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:01.502575 (+ 388us) server_negotiation.cc:299] Negotiation successful
0809 02:20:01.502709 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":523,"thread_start_us":195,"threads_started":1}
I20260809 02:20:01.503249 14047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:01.496623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.130:45931 (local address 127.0.0.1:55402)
0809 02:20:01.497849 (+ 1226us) negotiation.cc:107] Waiting for socket to connect
0809 02:20:01.497864 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:20:01.497967 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:20:01.498743 (+ 776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:20:01.498748 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:20:01.498769 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:20:01.498844 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:20:01.498849 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:20:01.500198 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:20:01.500201 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:20:01.501324 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:20:01.501332 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:20:01.502818 (+ 1486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:20:01.502824 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:20:01.502928 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:01.502961 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 02:20:01.503037 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 02:20:01.503110 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1164,"thread_start_us":131,"threads_started":1}
I20260809 02:20:01.503775 14048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:01.497418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55402 (local address 127.15.75.130:45931)
0809 02:20:01.498258 (+ 840us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:01.498271 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:01.498291 (+ 20us) server_negotiation.cc:407] Connection header received
0809 02:20:01.498374 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:01.498379 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:01.498567 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:01.498712 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:20:01.498968 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:01.500075 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:01.501458 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:01.502257 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:01.502360 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:01.502529 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:01.503465 (+ 936us) server_negotiation.cc:299] Negotiation successful
0809 02:20:01.503595 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":731,"thread_start_us":132,"threads_started":1}
I20260809 02:20:01.505856 13923 tablet_service.cc:1467] Processing CreateTablet for tablet 4d8fc96d30424107af167822e43f8c36 (DEFAULT_TABLE table=test-table [id=f619b588fa8b444ea44cbf5be90b3f32]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:20:01.506362 13519 tablet_service.cc:1467] Processing CreateTablet for tablet 4d8fc96d30424107af167822e43f8c36 (DEFAULT_TABLE table=test-table [id=f619b588fa8b444ea44cbf5be90b3f32]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:20:01.506362 13717 tablet_service.cc:1467] Processing CreateTablet for tablet 4d8fc96d30424107af167822e43f8c36 (DEFAULT_TABLE table=test-table [id=f619b588fa8b444ea44cbf5be90b3f32]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:20:01.506868 13923 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d8fc96d30424107af167822e43f8c36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:20:01.507433 13717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d8fc96d30424107af167822e43f8c36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:20:01.507433 13519 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d8fc96d30424107af167822e43f8c36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:20:01.512998 14050 tablet_bootstrap.cc:492] T 4d8fc96d30424107af167822e43f8c36 P 7c2f822d288d4f5b9bfe85978573d75e: Bootstrap starting.
I20260809 02:20:01.514971 14051 tablet_bootstrap.cc:492] T 4d8fc96d30424107af167822e43f8c36 P 24c4ebc315d1493bb5c6c68a480e8750: Bootstrap starting.
I20260809 02:20:01.515228 14050 tablet_bootstrap.cc:654] T 4d8fc96d30424107af167822e43f8c36 P 7c2f822d288d4f5b9bfe85978573d75e: Neither blocks nor log segments found. Creating new log.
I20260809 02:20:01.515940 14050 log.cc:824] T 4d8fc96d30424107af167822e43f8c36 P 7c2f822d288d4f5b9bfe85978573d75e: Log is configured to *not* fsync() on all Append() calls
I20260809 02:20:01.516577 14053 tablet_bootstrap.cc:492] T 4d8fc96d30424107af167822e43f8c36 P fe2cbb6e2e274da8b21d4062683e5cef: Bootstrap starting.
I20260809 02:20:01.516925 14051 tablet_bootstrap.cc:654] T 4d8fc96d30424107af167822e43f8c36 P 24c4ebc315d1493bb5c6c68a480e8750: Neither blocks nor log segments found. Creating new log.
I20260809 02:20:01.517764 14051 log.cc:824] T 4d8fc96d30424107af167822e43f8c36 P 24c4ebc315d1493bb5c6c68a480e8750: Log is configured to *not* fsync() on all Append() calls
I20260809 02:20:01.518263 14050 tablet_bootstrap.cc:492] T 4d8fc96d30424107af167822e43f8c36 P 7c2f822d288d4f5b9bfe85978573d75e: No bootstrap required, opened a new log
I20260809 02:20:01.518457 14050 ts_tablet_manager.cc:1397] T 4d8fc96d30424107af167822e43f8c36 P 7c2f822d288d4f5b9bfe85978573d75e: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 02:20:01.518991 14053 tablet_bootstrap.cc:654] T 4d8fc96d30424107af167822e43f8c36 P fe2cbb6e2e274da8b21d4062683e5cef: Neither blocks nor log segments found. Creating new log.
I20260809 02:20:01.519470 14051 tablet_bootstrap.cc:492] T 4d8fc96d30424107af167822e43f8c36 P 24c4ebc315d1493bb5c6c68a480e8750: No bootstrap required, opened a new log
I20260809 02:20:01.519623 14051 ts_tablet_manager.cc:1397] T 4d8fc96d30424107af167822e43f8c36 P 24c4ebc315d1493bb5c6c68a480e8750: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260809 02:20:01.519749 14053 log.cc:824] T 4d8fc96d30424107af167822e43f8c36 P fe2cbb6e2e274da8b21d4062683e5cef: Log is configured to *not* fsync() on all Append() calls
I20260809 02:20:01.521754 14053 tablet_bootstrap.cc:492] T 4d8fc96d30424107af167822e43f8c36 P fe2cbb6e2e274da8b21d4062683e5cef: No bootstrap required, opened a new log
I20260809 02:20:01.521944 14053 ts_tablet_manager.cc:1397] T 4d8fc96d30424107af167822e43f8c36 P fe2cbb6e2e274da8b21d4062683e5cef: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 02:20:01.522809 14050 raft_consensus.cc:348] T 4d8fc96d30424107af167822e43f8c36 P 7c2f822d288d4f5b9bfe85978573d75e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c2f822d288d4f5b9bfe85978573d75e" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 38067 } } peers { permanent_uuid: "fe2cbb6e2e274da8b21d4062683e5cef" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45931 } } peers { permanent_uuid: "24c4ebc315d1493bb5c6c68a480e8750" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 33673 } }
I20260809 02:20:01.523090 14050 raft_consensus.cc:374] T 4d8fc96d30424107af167822e43f8c36 P 7c2f822d288d4f5b9bfe85978573d75e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:20:01.522950 14051 raft_consensus.cc:348] T 4d8fc96d30424107af167822e43f8c36 P 24c4ebc315d1493bb5c6c68a480e8750 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c2f822d288d4f5b9bfe85978573d75e" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 38067 } } peers { permanent_uuid: "fe2cbb6e2e274da8b21d4062683e5cef" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45931 } } peers { permanent_uuid: "24c4ebc315d1493bb5c6c68a480e8750" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 33673 } }
I20260809 02:20:01.523165 14050 raft_consensus.cc:729] T 4d8fc96d30424107af167822e43f8c36 P 7c2f822d288d4f5b9bfe85978573d75e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c2f822d288d4f5b9bfe85978573d75e, State: Initialized, Role: FOLLOWER
I20260809 02:20:01.523180 14051 raft_consensus.cc:374] T 4d8fc96d30424107af167822e43f8c36 P 24c4ebc315d1493bb5c6c68a480e8750 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:20:01.523262 14051 raft_consensus.cc:729] T 4d8fc96d30424107af167822e43f8c36 P 24c4ebc315d1493bb5c6c68a480e8750 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24c4ebc315d1493bb5c6c68a480e8750, State: Initialized, Role: FOLLOWER
I20260809 02:20:01.523631 14050 consensus_queue.cc:260] T 4d8fc96d30424107af167822e43f8c36 P 7c2f822d288d4f5b9bfe85978573d75e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c2f822d288d4f5b9bfe85978573d75e" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 38067 } } peers { permanent_uuid: "fe2cbb6e2e274da8b21d4062683e5cef" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45931 } } peers { permanent_uuid: "24c4ebc315d1493bb5c6c68a480e8750" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 33673 } }
I20260809 02:20:01.523692 14051 consensus_queue.cc:260] T 4d8fc96d30424107af167822e43f8c36 P 24c4ebc315d1493bb5c6c68a480e8750 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c2f822d288d4f5b9bfe85978573d75e" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 38067 } } peers { permanent_uuid: "fe2cbb6e2e274da8b21d4062683e5cef" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45931 } } peers { permanent_uuid: "24c4ebc315d1493bb5c6c68a480e8750" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 33673 } }
I20260809 02:20:01.524329 14029 heartbeater.cc:503] Master 127.15.75.190:42471 was elected leader, sending a full tablet report...
I20260809 02:20:01.524627 14050 ts_tablet_manager.cc:1428] T 4d8fc96d30424107af167822e43f8c36 P 7c2f822d288d4f5b9bfe85978573d75e: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 02:20:01.525266 14051 ts_tablet_manager.cc:1428] T 4d8fc96d30424107af167822e43f8c36 P 24c4ebc315d1493bb5c6c68a480e8750: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 02:20:01.525614 13608 heartbeater.cc:503] Master 127.15.75.190:42471 was elected leader, sending a full tablet report...
I20260809 02:20:01.526454 14053 raft_consensus.cc:348] T 4d8fc96d30424107af167822e43f8c36 P fe2cbb6e2e274da8b21d4062683e5cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c2f822d288d4f5b9bfe85978573d75e" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 38067 } } peers { permanent_uuid: "fe2cbb6e2e274da8b21d4062683e5cef" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45931 } } peers { permanent_uuid: "24c4ebc315d1493bb5c6c68a480e8750" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 33673 } }
I20260809 02:20:01.527436 14053 raft_consensus.cc:374] T 4d8fc96d30424107af167822e43f8c36 P fe2cbb6e2e274da8b21d4062683e5cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:20:01.527520 14053 raft_consensus.cc:729] T 4d8fc96d30424107af167822e43f8c36 P fe2cbb6e2e274da8b21d4062683e5cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe2cbb6e2e274da8b21d4062683e5cef, State: Initialized, Role: FOLLOWER
I20260809 02:20:01.527916 14053 consensus_queue.cc:260] T 4d8fc96d30424107af167822e43f8c36 P fe2cbb6e2e274da8b21d4062683e5cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c2f822d288d4f5b9bfe85978573d75e" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 38067 } } peers { permanent_uuid: "fe2cbb6e2e274da8b21d4062683e5cef" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45931 } } peers { permanent_uuid: "24c4ebc315d1493bb5c6c68a480e8750" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 33673 } }
I20260809 02:20:01.528564 13817 heartbeater.cc:503] Master 127.15.75.190:42471 was elected leader, sending a full tablet report...
I20260809 02:20:01.528880 14053 ts_tablet_manager.cc:1428] T 4d8fc96d30424107af167822e43f8c36 P fe2cbb6e2e274da8b21d4062683e5cef: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260809 02:20:01.568897 14062 raft_consensus.cc:482] T 4d8fc96d30424107af167822e43f8c36 P 7c2f822d288d4f5b9bfe85978573d75e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:20:01.569124 14062 raft_consensus.cc:504] T 4d8fc96d30424107af167822e43f8c36 P 7c2f822d288d4f5b9bfe85978573d75e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c2f822d288d4f5b9bfe85978573d75e" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 38067 } } peers { permanent_uuid: "fe2cbb6e2e274da8b21d4062683e5cef" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45931 } } peers { permanent_uuid: "24c4ebc315d1493bb5c6c68a480e8750" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 33673 } }
I20260809 02:20:01.570336 14062 leader_election.cc:288] T 4d8fc96d30424107af167822e43f8c36 P 7c2f822d288d4f5b9bfe85978573d75e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fe2cbb6e2e274da8b21d4062683e5cef (127.15.75.130:45931), 24c4ebc315d1493bb5c6c68a480e8750 (127.15.75.129:33673)
W20260809 02:20:01.571590 13612 tablet.cc:2369] T 4d8fc96d30424107af167822e43f8c36 P 24c4ebc315d1493bb5c6c68a480e8750: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:20:01.575001 14065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:01.570572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.129:33673 (local address 127.15.75.131:40735)
0809 02:20:01.570950 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0809 02:20:01.570964 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:20:01.571069 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:20:01.571503 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:20:01.571506 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:20:01.571522 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:20:01.571645 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:20:01.571653 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:20:01.572485 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:20:01.572487 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:20:01.573893 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:20:01.573905 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:20:01.574564 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:20:01.574568 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:20:01.574671 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:01.574701 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 02:20:01.574762 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 02:20:01.574815 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"mutex_wait_us":66,"thread_start_us":134,"threads_started":1}
I20260809 02:20:01.575037 14049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:01.570754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.131:40735 (local address 127.15.75.129:33673)
0809 02:20:01.570928 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:01.570933 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:01.571058 (+ 125us) server_negotiation.cc:407] Connection header received
0809 02:20:01.571207 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:01.571212 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:01.571274 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:01.571338 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:20:01.571785 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:01.572334 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:01.574047 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:01.574428 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:01.574581 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:01.574635 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:01.574828 (+ 193us) server_negotiation.cc:299] Negotiation successful
0809 02:20:01.574913 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 02:20:01.575147 14048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:01.570597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.75.131:55291 (local address 127.15.75.130:45931)
0809 02:20:01.570721 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:01.570726 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:01.570928 (+ 202us) server_negotiation.cc:407] Connection header received
0809 02:20:01.570994 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:01.570999 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:01.571058 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:01.571122 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:20:01.571526 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:01.572329 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:01.574067 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:01.574445 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:01.574585 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:01.574636 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:01.574900 (+ 264us) server_negotiation.cc:299] Negotiation successful
0809 02:20:01.575000 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 02:20:01.575037 14032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:01.570518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.130:45931 (local address 127.15.75.131:55291)
0809 02:20:01.570721 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0809 02:20:01.570736 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:20:01.570834 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:20:01.571172 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:20:01.571177 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:20:01.571202 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:20:01.571340 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:20:01.571349 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:20:01.572451 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:20:01.572454 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:20:01.573901 (+ 1447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:20:01.573906 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:20:01.574625 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:20:01.574629 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:20:01.574735 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:01.574780 (+ 45us) client_negotiation.cc:769] Sending connection context
0809 02:20:01.574864 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 02:20:01.574933 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"mutex_wait_us":132}
I20260809 02:20:01.575876 13550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d8fc96d30424107af167822e43f8c36" candidate_uuid: "7c2f822d288d4f5b9bfe85978573d75e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c4ebc315d1493bb5c6c68a480e8750" is_pre_election: true
I20260809 02:20:01.576064 13747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d8fc96d30424107af167822e43f8c36" candidate_uuid: "7c2f822d288d4f5b9bfe85978573d75e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe2cbb6e2e274da8b21d4062683e5cef" is_pre_election: true
I20260809 02:20:01.576192 13550 raft_consensus.cc:2454] T 4d8fc96d30424107af167822e43f8c36 P 24c4ebc315d1493bb5c6c68a480e8750 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c2f822d288d4f5b9bfe85978573d75e in term 0.
I20260809 02:20:01.576318 13747 raft_consensus.cc:2454] T 4d8fc96d30424107af167822e43f8c36 P fe2cbb6e2e274da8b21d4062683e5cef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c2f822d288d4f5b9bfe85978573d75e in term 0.
I20260809 02:20:01.576771 13841 leader_election.cc:302] T 4d8fc96d30424107af167822e43f8c36 P 7c2f822d288d4f5b9bfe85978573d75e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 24c4ebc315d1493bb5c6c68a480e8750, 7c2f822d288d4f5b9bfe85978573d75e, fe2cbb6e2e274da8b21d4062683e5cef; no voters:
I20260809 02:20:01.577075 14062 raft_consensus.cc:2781] T 4d8fc96d30424107af167822e43f8c36 P 7c2f822d288d4f5b9bfe85978573d75e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:20:01.577148 14062 raft_consensus.cc:482] T 4d8fc96d30424107af167822e43f8c36 P 7c2f822d288d4f5b9bfe85978573d75e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:20:01.577227 14062 raft_consensus.cc:3037] T 4d8fc96d30424107af167822e43f8c36 P 7c2f822d288d4f5b9bfe85978573d75e [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:20:01.578328 14062 raft_consensus.cc:504] T 4d8fc96d30424107af167822e43f8c36 P 7c2f822d288d4f5b9bfe85978573d75e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c2f822d288d4f5b9bfe85978573d75e" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 38067 } } peers { permanent_uuid: "fe2cbb6e2e274da8b21d4062683e5cef" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45931 } } peers { permanent_uuid: "24c4ebc315d1493bb5c6c68a480e8750" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 33673 } }
I20260809 02:20:01.578665 14062 leader_election.cc:288] T 4d8fc96d30424107af167822e43f8c36 P 7c2f822d288d4f5b9bfe85978573d75e [CANDIDATE]: Term 1 election: Requested vote from peers fe2cbb6e2e274da8b21d4062683e5cef (127.15.75.130:45931), 24c4ebc315d1493bb5c6c68a480e8750 (127.15.75.129:33673)
I20260809 02:20:01.579025 13747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d8fc96d30424107af167822e43f8c36" candidate_uuid: "7c2f822d288d4f5b9bfe85978573d75e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe2cbb6e2e274da8b21d4062683e5cef"
I20260809 02:20:01.579025 13550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d8fc96d30424107af167822e43f8c36" candidate_uuid: "7c2f822d288d4f5b9bfe85978573d75e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c4ebc315d1493bb5c6c68a480e8750"
I20260809 02:20:01.579144 13747 raft_consensus.cc:3037] T 4d8fc96d30424107af167822e43f8c36 P fe2cbb6e2e274da8b21d4062683e5cef [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:20:01.579144 13550 raft_consensus.cc:3037] T 4d8fc96d30424107af167822e43f8c36 P 24c4ebc315d1493bb5c6c68a480e8750 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:20:01.580405 13550 raft_consensus.cc:2454] T 4d8fc96d30424107af167822e43f8c36 P 24c4ebc315d1493bb5c6c68a480e8750 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c2f822d288d4f5b9bfe85978573d75e in term 1.
I20260809 02:20:01.580405 13747 raft_consensus.cc:2454] T 4d8fc96d30424107af167822e43f8c36 P fe2cbb6e2e274da8b21d4062683e5cef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c2f822d288d4f5b9bfe85978573d75e in term 1.
I20260809 02:20:01.580735 13841 leader_election.cc:302] T 4d8fc96d30424107af167822e43f8c36 P 7c2f822d288d4f5b9bfe85978573d75e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 24c4ebc315d1493bb5c6c68a480e8750, 7c2f822d288d4f5b9bfe85978573d75e, fe2cbb6e2e274da8b21d4062683e5cef; no voters:
I20260809 02:20:01.580921 14062 raft_consensus.cc:2781] T 4d8fc96d30424107af167822e43f8c36 P 7c2f822d288d4f5b9bfe85978573d75e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:20:01.581146 14062 raft_consensus.cc:686] T 4d8fc96d30424107af167822e43f8c36 P 7c2f822d288d4f5b9bfe85978573d75e [term 1 LEADER]: Becoming Leader. State: Replica: 7c2f822d288d4f5b9bfe85978573d75e, State: Running, Role: LEADER
I20260809 02:20:01.581477 14062 consensus_queue.cc:237] T 4d8fc96d30424107af167822e43f8c36 P 7c2f822d288d4f5b9bfe85978573d75e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c2f822d288d4f5b9bfe85978573d75e" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 38067 } } peers { permanent_uuid: "fe2cbb6e2e274da8b21d4062683e5cef" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45931 } } peers { permanent_uuid: "24c4ebc315d1493bb5c6c68a480e8750" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 33673 } }
I20260809 02:20:01.584415 13343 catalog_manager.cc:5495] T 4d8fc96d30424107af167822e43f8c36 P 7c2f822d288d4f5b9bfe85978573d75e reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c2f822d288d4f5b9bfe85978573d75e (127.15.75.131). New cstate: current_term: 1 leader_uuid: "7c2f822d288d4f5b9bfe85978573d75e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c2f822d288d4f5b9bfe85978573d75e" member_type: VOTER last_known_addr { host: "127.15.75.131" port: 38067 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fe2cbb6e2e274da8b21d4062683e5cef" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45931 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24c4ebc315d1493bb5c6c68a480e8750" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 33673 } health_report { overall_health: UNKNOWN } } }
I20260809 02:20:01.598600 14045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:01.595268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42726 (local address 127.15.75.131:38067)
0809 02:20:01.595389 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:01.595392 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:01.595427 (+ 35us) server_negotiation.cc:407] Connection header received
0809 02:20:01.595573 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:01.595575 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:01.595623 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:01.595709 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:20:01.595925 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:01.596398 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:01.597294 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:01.597563 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:01.597655 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:01.597904 (+ 249us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:20:01.598289 (+ 385us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:20:01.598386 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:01.598441 (+ 55us) server_negotiation.cc:299] Negotiation successful
0809 02:20:01.598486 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 02:20:01.604453 13550 raft_consensus.cc:1261] T 4d8fc96d30424107af167822e43f8c36 P 24c4ebc315d1493bb5c6c68a480e8750 [term 1 FOLLOWER]: Refusing update from remote peer 7c2f822d288d4f5b9bfe85978573d75e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:20:01.604452 13747 raft_consensus.cc:1261] T 4d8fc96d30424107af167822e43f8c36 P fe2cbb6e2e274da8b21d4062683e5cef [term 1 FOLLOWER]: Refusing update from remote peer 7c2f822d288d4f5b9bfe85978573d75e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:20:01.605316 14062 consensus_queue.cc:1035] T 4d8fc96d30424107af167822e43f8c36 P 7c2f822d288d4f5b9bfe85978573d75e [LEADER]: Connected to new peer: Peer: permanent_uuid: "24c4ebc315d1493bb5c6c68a480e8750" member_type: VOTER last_known_addr { host: "127.15.75.129" port: 33673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:20:01.605605 14067 consensus_queue.cc:1035] T 4d8fc96d30424107af167822e43f8c36 P 7c2f822d288d4f5b9bfe85978573d75e [LEADER]: Connected to new peer: Peer: permanent_uuid: "fe2cbb6e2e274da8b21d4062683e5cef" member_type: VOTER last_known_addr { host: "127.15.75.130" port: 45931 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:20:01.613692 14073 mvcc.cc:205] Tried to move back new op lower bound from 7316447238563885056 to 7316447238480416768. Current Snapshot: MvccSnapshot[applied={T|T < 7316447238563885056}]
I20260809 02:20:01.621616 13345 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49872:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:20:01.621829 13345 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49872:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:20:01.624282 13345 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 86196f16428e4df0a5a3072eb29b1ecb: Sending DeleteTablet for 3 replicas of tablet 4d8fc96d30424107af167822e43f8c36
I20260809 02:20:01.625191 13519 tablet_service.cc:1514] Processing DeleteTablet for tablet 4d8fc96d30424107af167822e43f8c36 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:20:01 UTC) from {username='slave'} at 127.0.0.1:45996
I20260809 02:20:01.625170 13717 tablet_service.cc:1514] Processing DeleteTablet for tablet 4d8fc96d30424107af167822e43f8c36 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:20:01 UTC) from {username='slave'} at 127.0.0.1:55402
I20260809 02:20:01.625204 13923 tablet_service.cc:1514] Processing DeleteTablet for tablet 4d8fc96d30424107af167822e43f8c36 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:20:01 UTC) from {username='slave'} at 127.0.0.1:42710
I20260809 02:20:01.625779 14087 tablet_replica.cc:330] T 4d8fc96d30424107af167822e43f8c36 P 24c4ebc315d1493bb5c6c68a480e8750: stopping tablet replica
I20260809 02:20:01.625816 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 13424
I20260809 02:20:01.625674 14085 tablet_replica.cc:330] T 4d8fc96d30424107af167822e43f8c36 P 7c2f822d288d4f5b9bfe85978573d75e: stopping tablet replica
I20260809 02:20:01.631217 14085 raft_consensus.cc:2229] T 4d8fc96d30424107af167822e43f8c36 P 7c2f822d288d4f5b9bfe85978573d75e [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:20:01.631608 14085 raft_consensus.cc:2258] T 4d8fc96d30424107af167822e43f8c36 P 7c2f822d288d4f5b9bfe85978573d75e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:20:01.633891 14085 ts_tablet_manager.cc:1905] T 4d8fc96d30424107af167822e43f8c36 P 7c2f822d288d4f5b9bfe85978573d75e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:20:01.637467 14085 ts_tablet_manager.cc:1918] T 4d8fc96d30424107af167822e43f8c36 P 7c2f822d288d4f5b9bfe85978573d75e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:20:01.637580 14085 log.cc:1196] T 4d8fc96d30424107af167822e43f8c36 P 7c2f822d288d4f5b9bfe85978573d75e: Deleting WAL directory at /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241741788589-15662-0/minicluster-data/ts-2/wal/wals/4d8fc96d30424107af167822e43f8c36
I20260809 02:20:01.637934 14085 ts_tablet_manager.cc:1939] T 4d8fc96d30424107af167822e43f8c36 P 7c2f822d288d4f5b9bfe85978573d75e: Deleting consensus metadata
I20260809 02:20:01.639226 13325 catalog_manager.cc:4841] TS 7c2f822d288d4f5b9bfe85978573d75e (127.15.75.131:38067): tablet 4d8fc96d30424107af167822e43f8c36 (table test-table [id=f619b588fa8b444ea44cbf5be90b3f32]) successfully deleted
W20260809 02:20:01.643034 13323 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.75.129:33673 (error 108)
I20260809 02:20:01.643249 14086 tablet_replica.cc:330] T 4d8fc96d30424107af167822e43f8c36 P fe2cbb6e2e274da8b21d4062683e5cef: stopping tablet replica
I20260809 02:20:01.643589 14086 raft_consensus.cc:2229] T 4d8fc96d30424107af167822e43f8c36 P fe2cbb6e2e274da8b21d4062683e5cef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:20:01.643946 14086 raft_consensus.cc:2258] T 4d8fc96d30424107af167822e43f8c36 P fe2cbb6e2e274da8b21d4062683e5cef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:20:01.643965 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 13615
I20260809 02:20:01.659363 14044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:01.643892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.129:33673 (local address 127.0.0.1:46012)
0809 02:20:01.659162 (+ 15270us) negotiation.cc:107] Waiting for socket to connect
0809 02:20:01.659257 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.129:33673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15157}
W20260809 02:20:01.666178 13323 catalog_manager.cc:4584] TS 24c4ebc315d1493bb5c6c68a480e8750 (127.15.75.129:33673): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4d8fc96d30424107af167822e43f8c36: Network error: Client connection negotiation failed: client connection to 127.15.75.129:33673: connect: Connection refused (error 111)
I20260809 02:20:01.672312 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 13822
I20260809 02:20:01.677632 14044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:01.672535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.130:45931 (local address 127.0.0.1:55412)
0809 02:20:01.677452 (+ 4917us) negotiation.cc:107] Waiting for socket to connect
0809 02:20:01.677525 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.130:45931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4802}
I20260809 02:20:01.683039 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 13299
2026-08-09T02:20:01Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1083 ms)
[----------] 6 tests from EncryptionPolicyTest (10866 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241741788589-15662-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:20:01 dist-test-slave-s11t krb5kdc[14092](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:20:01 dist-test-slave-s11t krb5kdc[14092](info): set up 2 sockets
Aug 09 02:20:01 dist-test-slave-s11t krb5kdc[14092](info): commencing operation
krb5kdc: starting...
W20260809 02:20:02.925436 15662 mini_kdc.cc:121] Time spent starting KDC: real 1.211s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Aug 09 02:20:03 dist-test-slave-s11t krb5kdc[14092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786242003, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:20:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:20:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241741788589-15662-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241741788589-15662-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 02:20:03.056304 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241741788589-15662-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241741788589-15662-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241741788589-15662-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41491
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241741788589-15662-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41491
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:20:03.170150 14111 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:20:03.170403 14111 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:20:03.170441 14111 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:20:03.174315 14111 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:20:03.174358 14111 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:20:03.174377 14111 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:20:03.174393 14111 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:20:03.174408 14111 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:20:03.179459 14111 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241741788589-15662-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241741788589-15662-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41491
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241741788589-15662-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41491
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14111
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241741788589-15662-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:20:03.180346 14111 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:20:03.181293 14111 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 02:20:03.189419 14111 server_base.cc:1034] running on GCE node
W20260809 02:20:03.189404 14117 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:20:03.189388 14118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:20:03.189375 14120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:20:03.190321 14111 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:20:03.191324 14111 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:20:03.192545 14111 hybrid_clock.cc:648] HybridClock initialized: now 1786242003192531 us; error 33 us; skew 500 ppm
Aug 09 02:20:03 dist-test-slave-s11t krb5kdc[14092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786242003, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:20:03.195258 14111 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 02:20:03.196133 14111 webserver.cc:460] Webserver started at http://127.0.0.1:39597/ using document root <none> and password file <none>
I20260809 02:20:03.196663 14111 fs_manager.cc:359] Metadata directory not provided
I20260809 02:20:03.196846 14111 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:20:03.197075 14111 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:20:03.198995 14111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241741788589-15662-0/minicluster-data/master-0/data/instance:
uuid: "d1d5167a61854bfb8626aacdff375159"
format_stamp: "Formatted at 2026-08-09 02:20:03 on dist-test-slave-s11t"
server_key: "0be76814d4dbaa849e28ccacac3984ca"
server_key_iv: "c313a83b211f7bbdc18a8dd288d78df2"
server_key_version: "encryptionkey@0"
I20260809 02:20:03.199544 14111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241741788589-15662-0/minicluster-data/master-0/wal/instance:
uuid: "d1d5167a61854bfb8626aacdff375159"
format_stamp: "Formatted at 2026-08-09 02:20:03 on dist-test-slave-s11t"
server_key: "0be76814d4dbaa849e28ccacac3984ca"
server_key_iv: "c313a83b211f7bbdc18a8dd288d78df2"
server_key_version: "encryptionkey@0"
I20260809 02:20:03.203097 14111 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260809 02:20:03.205368 14127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:20:03.206451 14111 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 02:20:03.206590 14111 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241741788589-15662-0/minicluster-data/master-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241741788589-15662-0/minicluster-data/master-0/wal
uuid: "d1d5167a61854bfb8626aacdff375159"
format_stamp: "Formatted at 2026-08-09 02:20:03 on dist-test-slave-s11t"
server_key: "0be76814d4dbaa849e28ccacac3984ca"
server_key_iv: "c313a83b211f7bbdc18a8dd288d78df2"
server_key_version: "encryptionkey@0"
I20260809 02:20:03.206696 14111 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241741788589-15662-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241741788589-15662-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241741788589-15662-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:20:03.217835 14111 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:20:03.220295 14111 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:20:03.220496 14111 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:20:03.227377 14111 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41491
I20260809 02:20:03.227411 14206 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41491 every 8 connection(s)
I20260809 02:20:03.228259 14111 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
I20260809 02:20:03.230861 14207 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:20:03.232761 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 14111
I20260809 02:20:03.232918 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241741788589-15662-0/minicluster-data/master-0/wal/instance
I20260809 02:20:03.233230 15662 external_mini_cluster.cc:1428] Setting key 21cd423efef180aeb402e6868613aee0
I20260809 02:20:03.236604 14207 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1d5167a61854bfb8626aacdff375159: Bootstrap starting.
I20260809 02:20:03.238734 14207 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d1d5167a61854bfb8626aacdff375159: Neither blocks nor log segments found. Creating new log.
I20260809 02:20:03.239465 14207 log.cc:824] T 00000000000000000000000000000000 P d1d5167a61854bfb8626aacdff375159: Log is configured to *not* fsync() on all Append() calls
I20260809 02:20:03.241524 14207 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1d5167a61854bfb8626aacdff375159: No bootstrap required, opened a new log
Aug 09 02:20:03 dist-test-slave-s11t krb5kdc[14092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786242003, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 02:20:03.245090 14207 raft_consensus.cc:348] T 00000000000000000000000000000000 P d1d5167a61854bfb8626aacdff375159 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1d5167a61854bfb8626aacdff375159" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41491 } }
I20260809 02:20:03.245323 14207 raft_consensus.cc:374] T 00000000000000000000000000000000 P d1d5167a61854bfb8626aacdff375159 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:20:03.245398 14207 raft_consensus.cc:729] T 00000000000000000000000000000000 P d1d5167a61854bfb8626aacdff375159 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1d5167a61854bfb8626aacdff375159, State: Initialized, Role: FOLLOWER
I20260809 02:20:03.245872 14207 consensus_queue.cc:260] T 00000000000000000000000000000000 P d1d5167a61854bfb8626aacdff375159 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1d5167a61854bfb8626aacdff375159" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41491 } }
I20260809 02:20:03.246069 14207 raft_consensus.cc:388] T 00000000000000000000000000000000 P d1d5167a61854bfb8626aacdff375159 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:20:03.246150 14207 raft_consensus.cc:482] T 00000000000000000000000000000000 P d1d5167a61854bfb8626aacdff375159 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:20:03.246259 14207 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d1d5167a61854bfb8626aacdff375159 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:20:03.247287 14207 raft_consensus.cc:504] T 00000000000000000000000000000000 P d1d5167a61854bfb8626aacdff375159 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1d5167a61854bfb8626aacdff375159" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41491 } }
I20260809 02:20:03.247615 14207 leader_election.cc:302] T 00000000000000000000000000000000 P d1d5167a61854bfb8626aacdff375159 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d1d5167a61854bfb8626aacdff375159; no voters:
I20260809 02:20:03.247915 14207 leader_election.cc:288] T 00000000000000000000000000000000 P d1d5167a61854bfb8626aacdff375159 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:20:03.248016 14214 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d1d5167a61854bfb8626aacdff375159 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:20:03.248232 14214 raft_consensus.cc:686] T 00000000000000000000000000000000 P d1d5167a61854bfb8626aacdff375159 [term 1 LEADER]: Becoming Leader. State: Replica: d1d5167a61854bfb8626aacdff375159, State: Running, Role: LEADER
I20260809 02:20:03.248487 14212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:03.234609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49561 (local address 127.0.0.1:41491)
0809 02:20:03.235027 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:03.235042 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:03.235075 (+ 33us) server_negotiation.cc:407] Connection header received
0809 02:20:03.235789 (+ 714us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:03.235808 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:03.236124 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:03.236458 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:03.237312 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:03.238266 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:03.239308 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:03.239817 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:03.239998 (+ 181us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:03.242811 (+ 2813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:20:03.242816 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:20:03.242829 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:20:03.242855 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:20:03.245187 (+ 2332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:03.245780 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:03.245787 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:03.245793 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:03.245906 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:03.246318 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:03.246320 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:03.246322 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:03.246727 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:20:03.246905 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:03.247231 (+ 326us) server_negotiation.cc:299] Negotiation successful
0809 02:20:03.247458 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":130,"threads_started":1}
I20260809 02:20:03.248538 14214 consensus_queue.cc:237] T 00000000000000000000000000000000 P d1d5167a61854bfb8626aacdff375159 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1d5167a61854bfb8626aacdff375159" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41491 } }
I20260809 02:20:03.249181 14207 sys_catalog.cc:564] T 00000000000000000000000000000000 P d1d5167a61854bfb8626aacdff375159 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:20:03.249702 14214 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1d5167a61854bfb8626aacdff375159 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d1d5167a61854bfb8626aacdff375159. Latest consensus state: current_term: 1 leader_uuid: "d1d5167a61854bfb8626aacdff375159" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1d5167a61854bfb8626aacdff375159" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41491 } } }
I20260809 02:20:03.249838 14214 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1d5167a61854bfb8626aacdff375159 [sys.catalog]: This master's current role is: LEADER
I20260809 02:20:03.250212 14222 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:20:03.250684 14215 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1d5167a61854bfb8626aacdff375159 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d1d5167a61854bfb8626aacdff375159" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1d5167a61854bfb8626aacdff375159" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41491 } } }
I20260809 02:20:03.250782 14215 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1d5167a61854bfb8626aacdff375159 [sys.catalog]: This master's current role is: LEADER
I20260809 02:20:03.251965 15662 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:20:03.254180 14222 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:20:03.260540 14222 catalog_manager.cc:1347] Generated new cluster ID: 2089a970c65c4801a760438b213a77db
I20260809 02:20:03.260608 14222 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:20:03.268265 14222 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:20:03.269521 14222 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:20:03.269995 14212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:03.254270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43267 (local address 127.0.0.1:41491)
0809 02:20:03.254379 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:03.254383 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:03.256548 (+ 2165us) server_negotiation.cc:407] Connection header received
0809 02:20:03.256650 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:03.256653 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:03.256721 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:03.256799 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:03.257697 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:03.258743 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:03.259622 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:03.259883 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:03.260001 (+ 118us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:03.265862 (+ 5861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:20:03.265866 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:20:03.265869 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:20:03.265904 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:20:03.267896 (+ 1992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:03.268475 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:03.268479 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:03.268480 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:03.268564 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:03.268977 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:03.268981 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:03.268982 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:03.269167 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:20:03.269296 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:03.269753 (+ 457us) server_negotiation.cc:299] Negotiation successful
0809 02:20:03.269851 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 02:20:03.286034 14222 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d1d5167a61854bfb8626aacdff375159: Generated new TSK 0
I20260809 02:20:03.286676 14222 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:20:03.292286 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 14111
W20260809 02:20:03.313412 14090 connection.cc:535] client connection to 127.0.0.1:41491 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41491): Connection reset by peer (error 104)
2026-08-09T02:20:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1614 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241741788589-15662-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:20:03 dist-test-slave-s11t krb5kdc[14252](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:20:03 dist-test-slave-s11t krb5kdc[14252](info): set up 2 sockets
Aug 09 02:20:03 dist-test-slave-s11t krb5kdc[14252](info): commencing operation
krb5kdc: starting...
W20260809 02:20:04.531687 15662 mini_kdc.cc:121] Time spent starting KDC: real 1.204s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Aug 09 02:20:04 dist-test-slave-s11t krb5kdc[14252](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786242004, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:20:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:20:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241741788589-15662-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241741788589-15662-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 02:20:04.655669 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241741788589-15662-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241741788589-15662-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241741788589-15662-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46583
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241741788589-15662-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46583
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:20:04.766079 14276 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:20:04.766381 14276 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:20:04.766463 14276 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:20:04.770364 14276 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:20:04.770450 14276 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:20:04.770489 14276 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:20:04.770550 14276 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:20:04.770589 14276 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:20:04.775581 14276 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241741788589-15662-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241741788589-15662-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46583
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241741788589-15662-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46583
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14276
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241741788589-15662-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:20:04.776566 14276 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:20:04.777560 14276 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:20:04.786029 14283 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:20:04.786043 14284 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:20:04.786053 14287 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:20:04.786353 14276 server_base.cc:1034] running on GCE node
I20260809 02:20:04.786898 14276 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:20:04.788000 14276 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:20:04.789233 14276 hybrid_clock.cc:648] HybridClock initialized: now 1786242004789193 us; error 44 us; skew 500 ppm
Aug 09 02:20:04 dist-test-slave-s11t krb5kdc[14252](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786242004, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:20:04.791939 14276 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 02:20:04.792789 14276 webserver.cc:460] Webserver started at http://127.0.0.1:40901/ using document root <none> and password file <none>
I20260809 02:20:04.793296 14276 fs_manager.cc:359] Metadata directory not provided
I20260809 02:20:04.793355 14276 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:20:04.793510 14276 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:20:04.795433 14276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241741788589-15662-0/minicluster-data/master-0/data/instance:
uuid: "817c085a77b441b1ab799cc5b72b75f3"
format_stamp: "Formatted at 2026-08-09 02:20:04 on dist-test-slave-s11t"
server_key: "84fc5699ba60e92345f4f41adb01e925"
server_key_iv: "75d1039d949a0a702b37ed77ec2cad36"
server_key_version: "encryptionkey@0"
I20260809 02:20:04.795930 14276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241741788589-15662-0/minicluster-data/master-0/wal/instance:
uuid: "817c085a77b441b1ab799cc5b72b75f3"
format_stamp: "Formatted at 2026-08-09 02:20:04 on dist-test-slave-s11t"
server_key: "84fc5699ba60e92345f4f41adb01e925"
server_key_iv: "75d1039d949a0a702b37ed77ec2cad36"
server_key_version: "encryptionkey@0"
I20260809 02:20:04.799456 14276 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260809 02:20:04.801797 14296 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:20:04.803131 14276 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 02:20:04.803270 14276 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241741788589-15662-0/minicluster-data/master-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241741788589-15662-0/minicluster-data/master-0/wal
uuid: "817c085a77b441b1ab799cc5b72b75f3"
format_stamp: "Formatted at 2026-08-09 02:20:04 on dist-test-slave-s11t"
server_key: "84fc5699ba60e92345f4f41adb01e925"
server_key_iv: "75d1039d949a0a702b37ed77ec2cad36"
server_key_version: "encryptionkey@0"
I20260809 02:20:04.803367 14276 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241741788589-15662-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241741788589-15662-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241741788589-15662-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:20:04.825879 14276 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:20:04.828903 14276 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:20:04.829094 14276 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:20:04.836376 14276 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46583
I20260809 02:20:04.836392 14374 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46583 every 8 connection(s)
I20260809 02:20:04.837360 14276 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
I20260809 02:20:04.840076 14375 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:20:04.841583 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 14276
I20260809 02:20:04.841746 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241741788589-15662-0/minicluster-data/master-0/wal/instance
I20260809 02:20:04.842072 15662 external_mini_cluster.cc:1428] Setting key aed67cb3904ac3096fdede30f12bc30f
I20260809 02:20:04.846246 14375 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 817c085a77b441b1ab799cc5b72b75f3: Bootstrap starting.
I20260809 02:20:04.848452 14375 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 817c085a77b441b1ab799cc5b72b75f3: Neither blocks nor log segments found. Creating new log.
I20260809 02:20:04.849242 14375 log.cc:824] T 00000000000000000000000000000000 P 817c085a77b441b1ab799cc5b72b75f3: Log is configured to *not* fsync() on all Append() calls
I20260809 02:20:04.850747 14378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:04.843396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37889 (local address 127.0.0.1:46583)
0809 02:20:04.843830 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:04.843845 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:04.843879 (+ 34us) server_negotiation.cc:407] Connection header received
0809 02:20:04.844475 (+ 596us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:04.844490 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:04.844874 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:04.845185 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:04.845579 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:04.846549 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:04.847442 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:04.847906 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:04.848089 (+ 183us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:04.848972 (+ 883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:20:04.848976 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:20:04.848988 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:20:04.849341 (+ 353us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:20:04.849432 (+ 91us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:20:04.849458 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:20:04.849595 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:04.849753 (+ 158us) server_negotiation.cc:299] Negotiation successful
0809 02:20:04.849882 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":137,"threads_started":1}
I20260809 02:20:04.851394 14375 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 817c085a77b441b1ab799cc5b72b75f3: No bootstrap required, opened a new log
I20260809 02:20:04.853482 15662 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:20:04.855252 14375 raft_consensus.cc:348] T 00000000000000000000000000000000 P 817c085a77b441b1ab799cc5b72b75f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "817c085a77b441b1ab799cc5b72b75f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46583 } }
I20260809 02:20:04.855540 14375 raft_consensus.cc:374] T 00000000000000000000000000000000 P 817c085a77b441b1ab799cc5b72b75f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:20:04.855618 14375 raft_consensus.cc:729] T 00000000000000000000000000000000 P 817c085a77b441b1ab799cc5b72b75f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 817c085a77b441b1ab799cc5b72b75f3, State: Initialized, Role: FOLLOWER
I20260809 02:20:04.856097 14375 consensus_queue.cc:260] T 00000000000000000000000000000000 P 817c085a77b441b1ab799cc5b72b75f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "817c085a77b441b1ab799cc5b72b75f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46583 } }
I20260809 02:20:04.856274 14375 raft_consensus.cc:388] T 00000000000000000000000000000000 P 817c085a77b441b1ab799cc5b72b75f3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:20:04.856362 14375 raft_consensus.cc:482] T 00000000000000000000000000000000 P 817c085a77b441b1ab799cc5b72b75f3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:20:04.856472 14375 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 817c085a77b441b1ab799cc5b72b75f3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:20:04.857508 14375 raft_consensus.cc:504] T 00000000000000000000000000000000 P 817c085a77b441b1ab799cc5b72b75f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "817c085a77b441b1ab799cc5b72b75f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46583 } }
I20260809 02:20:04.857836 14375 leader_election.cc:302] T 00000000000000000000000000000000 P 817c085a77b441b1ab799cc5b72b75f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 817c085a77b441b1ab799cc5b72b75f3; no voters:
I20260809 02:20:04.858124 14375 leader_election.cc:288] T 00000000000000000000000000000000 P 817c085a77b441b1ab799cc5b72b75f3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:20:04.858245 14389 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 817c085a77b441b1ab799cc5b72b75f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:20:04.858492 14389 raft_consensus.cc:686] T 00000000000000000000000000000000 P 817c085a77b441b1ab799cc5b72b75f3 [term 1 LEADER]: Becoming Leader. State: Replica: 817c085a77b441b1ab799cc5b72b75f3, State: Running, Role: LEADER
I20260809 02:20:04.858821 14389 consensus_queue.cc:237] T 00000000000000000000000000000000 P 817c085a77b441b1ab799cc5b72b75f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "817c085a77b441b1ab799cc5b72b75f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46583 } }
I20260809 02:20:04.859445 14375 sys_catalog.cc:564] T 00000000000000000000000000000000 P 817c085a77b441b1ab799cc5b72b75f3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:20:04.860282 14378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:04.855494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45045 (local address 127.0.0.1:46583)
0809 02:20:04.855600 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:04.855605 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:04.855820 (+ 215us) server_negotiation.cc:407] Connection header received
0809 02:20:04.855933 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:04.855935 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:04.855980 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:04.856045 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:04.856365 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:04.857126 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:04.858209 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:04.858550 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:04.858690 (+ 140us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:04.859646 (+ 956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:20:04.859649 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:20:04.859652 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:20:04.859664 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:20:04.859688 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:20:04.859700 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:20:04.859813 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:04.860122 (+ 309us) server_negotiation.cc:299] Negotiation successful
0809 02:20:04.860156 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 02:20:04.860661 14390 sys_catalog.cc:455] T 00000000000000000000000000000000 P 817c085a77b441b1ab799cc5b72b75f3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "817c085a77b441b1ab799cc5b72b75f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "817c085a77b441b1ab799cc5b72b75f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46583 } } }
I20260809 02:20:04.861356 14390 sys_catalog.cc:458] T 00000000000000000000000000000000 P 817c085a77b441b1ab799cc5b72b75f3 [sys.catalog]: This master's current role is: LEADER
I20260809 02:20:04.861642 14408 sys_catalog.cc:455] T 00000000000000000000000000000000 P 817c085a77b441b1ab799cc5b72b75f3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 817c085a77b441b1ab799cc5b72b75f3. Latest consensus state: current_term: 1 leader_uuid: "817c085a77b441b1ab799cc5b72b75f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "817c085a77b441b1ab799cc5b72b75f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46583 } } }
I20260809 02:20:04.861937 14408 sys_catalog.cc:458] T 00000000000000000000000000000000 P 817c085a77b441b1ab799cc5b72b75f3 [sys.catalog]: This master's current role is: LEADER
W20260809 02:20:04.864311 14409 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 817c085a77b441b1ab799cc5b72b75f3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:20:04.864377 14409 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:20:04.864461 14410 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:20:04.865325 14410 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:20:04.870564 14410 catalog_manager.cc:1347] Generated new cluster ID: 8ab2c0f955584560862342bdf2897e09
I20260809 02:20:04.870625 14410 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:20:04.876579 14410 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:20:04.877669 14410 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:20:04.893705 14410 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 817c085a77b441b1ab799cc5b72b75f3: Generated new TSK 0
I20260809 02:20:04.894368 14410 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:20:04.904706 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 14276
W20260809 02:20:04.920439 14250 connection.cc:535] client connection to 127.0.0.1:46583 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46583): Connection reset by peer (error 104)
2026-08-09T02:20:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1607 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241741788589-15662-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:20:04 dist-test-slave-s11t krb5kdc[14420](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:20:04 dist-test-slave-s11t krb5kdc[14420](info): set up 2 sockets
Aug 09 02:20:04 dist-test-slave-s11t krb5kdc[14420](info): commencing operation
krb5kdc: starting...
W20260809 02:20:06.147349 15662 mini_kdc.cc:121] Time spent starting KDC: real 1.212s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Aug 09 02:20:06 dist-test-slave-s11t krb5kdc[14420](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786242006, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:20:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:20:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241741788589-15662-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241741788589-15662-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 02:20:06.276856 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241741788589-15662-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241741788589-15662-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241741788589-15662-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45721
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241741788589-15662-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45721
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:20:06.385947 14439 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:20:06.386255 14439 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:20:06.386335 14439 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:20:06.390267 14439 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:20:06.390358 14439 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:20:06.390398 14439 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:20:06.390455 14439 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:20:06.390496 14439 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:20:06.395596 14439 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241741788589-15662-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241741788589-15662-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45721
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241741788589-15662-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45721
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14439
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241741788589-15662-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:20:06.396620 14439 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:20:06.397650 14439 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:20:06.405990 14445 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:20:06.406001 14448 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:20:06.406001 14444 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:20:06.406232 14439 server_base.cc:1034] running on GCE node
I20260809 02:20:06.406774 14439 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:20:06.407882 14439 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:20:06.409075 14439 hybrid_clock.cc:648] HybridClock initialized: now 1786242006409057 us; error 32 us; skew 500 ppm
Aug 09 02:20:06 dist-test-slave-s11t krb5kdc[14420](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786242006, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:20:06.411948 14439 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 02:20:06.412883 14439 webserver.cc:460] Webserver started at http://127.0.0.1:43005/ using document root <none> and password file <none>
I20260809 02:20:06.413444 14439 fs_manager.cc:359] Metadata directory not provided
I20260809 02:20:06.413542 14439 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:20:06.413772 14439 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:20:06.415745 14439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241741788589-15662-0/minicluster-data/master-0/data/instance:
uuid: "2ade52c0da76447ca0f8c512781e5a68"
format_stamp: "Formatted at 2026-08-09 02:20:06 on dist-test-slave-s11t"
server_key: "ecb73f03025b7fef06027fc9aae45681"
server_key_iv: "1c6592e0e3d1894b69da60d9f67e93c0"
server_key_version: "encryptionkey@0"
I20260809 02:20:06.416320 14439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241741788589-15662-0/minicluster-data/master-0/wal/instance:
uuid: "2ade52c0da76447ca0f8c512781e5a68"
format_stamp: "Formatted at 2026-08-09 02:20:06 on dist-test-slave-s11t"
server_key: "ecb73f03025b7fef06027fc9aae45681"
server_key_iv: "1c6592e0e3d1894b69da60d9f67e93c0"
server_key_version: "encryptionkey@0"
I20260809 02:20:06.419905 14439 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260809 02:20:06.422297 14457 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:20:06.423514 14439 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 02:20:06.423659 14439 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241741788589-15662-0/minicluster-data/master-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241741788589-15662-0/minicluster-data/master-0/wal
uuid: "2ade52c0da76447ca0f8c512781e5a68"
format_stamp: "Formatted at 2026-08-09 02:20:06 on dist-test-slave-s11t"
server_key: "ecb73f03025b7fef06027fc9aae45681"
server_key_iv: "1c6592e0e3d1894b69da60d9f67e93c0"
server_key_version: "encryptionkey@0"
I20260809 02:20:06.423771 14439 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241741788589-15662-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241741788589-15662-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241741788589-15662-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:20:06.428267 14439 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:20:06.431497 14439 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:20:06.431695 14439 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:20:06.438772 14439 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45721
I20260809 02:20:06.438822 14541 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45721 every 8 connection(s)
I20260809 02:20:06.439697 14439 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
I20260809 02:20:06.442435 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 14439
I20260809 02:20:06.442580 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241741788589-15662-0/minicluster-data/master-0/wal/instance
I20260809 02:20:06.442654 14542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:20:06.442947 15662 external_mini_cluster.cc:1428] Setting key c69d1529287155c52c2855e380ce7cab
I20260809 02:20:06.448218 14547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:06.444236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44591 (local address 127.0.0.1:45721)
0809 02:20:06.444659 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:06.444678 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:06.444768 (+ 90us) server_negotiation.cc:407] Connection header received
0809 02:20:06.445519 (+ 751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:06.445543 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:06.445903 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:06.446257 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:06.446893 (+ 636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:20:06.446898 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:20:06.446913 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:20:06.446924 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:20:06.446981 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:20:06.447006 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:20:06.447150 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:06.447215 (+ 65us) server_negotiation.cc:299] Negotiation successful
0809 02:20:06.447354 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":112,"threads_started":1}
I20260809 02:20:06.450137 14542 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ade52c0da76447ca0f8c512781e5a68: Bootstrap starting.
I20260809 02:20:06.451215 15662 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:20:06.452591 14542 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2ade52c0da76447ca0f8c512781e5a68: Neither blocks nor log segments found. Creating new log.
I20260809 02:20:06.453426 14542 log.cc:824] T 00000000000000000000000000000000 P 2ade52c0da76447ca0f8c512781e5a68: Log is configured to *not* fsync() on all Append() calls
I20260809 02:20:06.455367 14547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:06.453325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54235 (local address 127.0.0.1:45721)
0809 02:20:06.453454 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:06.453459 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:06.454008 (+ 549us) server_negotiation.cc:407] Connection header received
0809 02:20:06.454183 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:06.454186 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:06.454224 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:06.454284 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:06.454959 (+ 675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:20:06.454962 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:20:06.454964 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:20:06.454974 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:20:06.454998 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:20:06.455008 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:20:06.455076 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:06.455196 (+ 120us) server_negotiation.cc:299] Negotiation successful
0809 02:20:06.455227 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 02:20:06.455698 14542 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ade52c0da76447ca0f8c512781e5a68: No bootstrap required, opened a new log
I20260809 02:20:06.459158 14542 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ade52c0da76447ca0f8c512781e5a68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ade52c0da76447ca0f8c512781e5a68" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45721 } }
I20260809 02:20:06.459396 14542 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2ade52c0da76447ca0f8c512781e5a68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:20:06.459458 14542 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ade52c0da76447ca0f8c512781e5a68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ade52c0da76447ca0f8c512781e5a68, State: Initialized, Role: FOLLOWER
I20260809 02:20:06.459905 14542 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ade52c0da76447ca0f8c512781e5a68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ade52c0da76447ca0f8c512781e5a68" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45721 } }
I20260809 02:20:06.460055 14542 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2ade52c0da76447ca0f8c512781e5a68 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:20:06.460125 14542 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2ade52c0da76447ca0f8c512781e5a68 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:20:06.460227 14542 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ade52c0da76447ca0f8c512781e5a68 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:20:06.461212 14542 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2ade52c0da76447ca0f8c512781e5a68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ade52c0da76447ca0f8c512781e5a68" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45721 } }
I20260809 02:20:06.461548 14542 leader_election.cc:302] T 00000000000000000000000000000000 P 2ade52c0da76447ca0f8c512781e5a68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2ade52c0da76447ca0f8c512781e5a68; no voters:
I20260809 02:20:06.461876 14542 leader_election.cc:288] T 00000000000000000000000000000000 P 2ade52c0da76447ca0f8c512781e5a68 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:20:06.461983 14559 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2ade52c0da76447ca0f8c512781e5a68 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:20:06.462245 14559 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2ade52c0da76447ca0f8c512781e5a68 [term 1 LEADER]: Becoming Leader. State: Replica: 2ade52c0da76447ca0f8c512781e5a68, State: Running, Role: LEADER
I20260809 02:20:06.462622 14559 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ade52c0da76447ca0f8c512781e5a68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ade52c0da76447ca0f8c512781e5a68" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45721 } }
I20260809 02:20:06.463143 14542 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ade52c0da76447ca0f8c512781e5a68 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:20:06.464285 14562 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ade52c0da76447ca0f8c512781e5a68 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ade52c0da76447ca0f8c512781e5a68. Latest consensus state: current_term: 1 leader_uuid: "2ade52c0da76447ca0f8c512781e5a68" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ade52c0da76447ca0f8c512781e5a68" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45721 } } }
I20260809 02:20:06.464423 14562 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ade52c0da76447ca0f8c512781e5a68 [sys.catalog]: This master's current role is: LEADER
I20260809 02:20:06.464751 14560 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ade52c0da76447ca0f8c512781e5a68 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2ade52c0da76447ca0f8c512781e5a68" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ade52c0da76447ca0f8c512781e5a68" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45721 } } }
I20260809 02:20:06.464854 14560 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ade52c0da76447ca0f8c512781e5a68 [sys.catalog]: This master's current role is: LEADER
I20260809 02:20:06.464843 14576 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:20:06.468398 14576 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:20:06.473989 14576 catalog_manager.cc:1347] Generated new cluster ID: 96d64eb5988247dab7952c7661b1000d
I20260809 02:20:06.474053 14576 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:20:06.480640 14576 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:20:06.480934 14576 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:20:06.484318 14576 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2ade52c0da76447ca0f8c512781e5a68: Generated new TSK 0
I20260809 02:20:06.484952 14576 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:20:06.498713 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 14439
2026-08-09T02:20:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1590 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241741788589-15662-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:20:06 dist-test-slave-s11t krb5kdc[14594](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:20:06 dist-test-slave-s11t krb5kdc[14594](info): set up 2 sockets
Aug 09 02:20:06 dist-test-slave-s11t krb5kdc[14594](info): commencing operation
krb5kdc: starting...
W20260809 02:20:07.725173 15662 mini_kdc.cc:121] Time spent starting KDC: real 1.200s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Aug 09 02:20:07 dist-test-slave-s11t krb5kdc[14594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786242007, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:20:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:20:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241741788589-15662-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241741788589-15662-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 02:20:07.847677 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241741788589-15662-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241741788589-15662-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241741788589-15662-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37097
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241741788589-15662-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37097
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:20:07.957898 14614 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:20:07.958163 14614 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:20:07.958231 14614 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:20:07.962358 14614 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:20:07.962450 14614 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:20:07.962497 14614 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:20:07.962554 14614 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:20:07.962594 14614 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:20:07.967746 14614 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241741788589-15662-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241741788589-15662-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37097
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241741788589-15662-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37097
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14614
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241741788589-15662-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:20:07.968815 14614 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:20:07.969827 14614 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:20:07.978168 14620 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:20:07.978144 14623 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:20:07.978132 14621 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:20:07.978437 14614 server_base.cc:1034] running on GCE node
I20260809 02:20:07.979357 14614 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:20:07.980881 14614 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:20:07.982080 14614 hybrid_clock.cc:648] HybridClock initialized: now 1786242007982062 us; error 35 us; skew 500 ppm
Aug 09 02:20:07 dist-test-slave-s11t krb5kdc[14594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786242007, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:20:07.989975 14614 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 02:20:07.990818 14614 webserver.cc:460] Webserver started at http://127.0.0.1:40907/ using document root <none> and password file <none>
I20260809 02:20:07.991340 14614 fs_manager.cc:359] Metadata directory not provided
I20260809 02:20:07.991398 14614 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:20:07.991559 14614 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:20:07.993643 14614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241741788589-15662-0/minicluster-data/master-0/data/instance:
uuid: "caa10ca8dd4d468397f3a8b2c1fd986a"
format_stamp: "Formatted at 2026-08-09 02:20:07 on dist-test-slave-s11t"
server_key: "c8436e22481944a6dcf6fc15a46afc24"
server_key_iv: "a3e8a56da86496d4102e5be8d18eb2fc"
server_key_version: "encryptionkey@0"
I20260809 02:20:07.994218 14614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241741788589-15662-0/minicluster-data/master-0/wal/instance:
uuid: "caa10ca8dd4d468397f3a8b2c1fd986a"
format_stamp: "Formatted at 2026-08-09 02:20:07 on dist-test-slave-s11t"
server_key: "c8436e22481944a6dcf6fc15a46afc24"
server_key_iv: "a3e8a56da86496d4102e5be8d18eb2fc"
server_key_version: "encryptionkey@0"
I20260809 02:20:07.997792 14614 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20260809 02:20:08.000036 14636 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:20:08.001272 14614 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 02:20:08.001405 14614 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241741788589-15662-0/minicluster-data/master-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241741788589-15662-0/minicluster-data/master-0/wal
uuid: "caa10ca8dd4d468397f3a8b2c1fd986a"
format_stamp: "Formatted at 2026-08-09 02:20:07 on dist-test-slave-s11t"
server_key: "c8436e22481944a6dcf6fc15a46afc24"
server_key_iv: "a3e8a56da86496d4102e5be8d18eb2fc"
server_key_version: "encryptionkey@0"
I20260809 02:20:08.001508 14614 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241741788589-15662-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241741788589-15662-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241741788589-15662-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:20:08.024338 14614 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:20:08.027673 14614 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:20:08.027881 14614 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:20:08.035007 14614 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37097
I20260809 02:20:08.035035 14714 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37097 every 8 connection(s)
I20260809 02:20:08.035930 14614 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
I20260809 02:20:08.038565 14716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:20:08.043954 14716 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P caa10ca8dd4d468397f3a8b2c1fd986a: Bootstrap starting.
I20260809 02:20:08.044579 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 14614
I20260809 02:20:08.044742 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241741788589-15662-0/minicluster-data/master-0/wal/instance
I20260809 02:20:08.045048 15662 external_mini_cluster.cc:1428] Setting key e269440862336e8cf6dcd63f8e40d60e
I20260809 02:20:08.046777 14716 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P caa10ca8dd4d468397f3a8b2c1fd986a: Neither blocks nor log segments found. Creating new log.
I20260809 02:20:08.047492 14716 log.cc:824] T 00000000000000000000000000000000 P caa10ca8dd4d468397f3a8b2c1fd986a: Log is configured to *not* fsync() on all Append() calls
I20260809 02:20:08.049624 14716 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P caa10ca8dd4d468397f3a8b2c1fd986a: No bootstrap required, opened a new log
Aug 09 02:20:08 dist-test-slave-s11t krb5kdc[14594](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786242007, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 02:20:08.054179 14716 raft_consensus.cc:348] T 00000000000000000000000000000000 P caa10ca8dd4d468397f3a8b2c1fd986a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caa10ca8dd4d468397f3a8b2c1fd986a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37097 } }
I20260809 02:20:08.054431 14716 raft_consensus.cc:374] T 00000000000000000000000000000000 P caa10ca8dd4d468397f3a8b2c1fd986a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:20:08.054510 14716 raft_consensus.cc:729] T 00000000000000000000000000000000 P caa10ca8dd4d468397f3a8b2c1fd986a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: caa10ca8dd4d468397f3a8b2c1fd986a, State: Initialized, Role: FOLLOWER
I20260809 02:20:08.055023 14716 consensus_queue.cc:260] T 00000000000000000000000000000000 P caa10ca8dd4d468397f3a8b2c1fd986a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caa10ca8dd4d468397f3a8b2c1fd986a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37097 } }
I20260809 02:20:08.055207 14716 raft_consensus.cc:388] T 00000000000000000000000000000000 P caa10ca8dd4d468397f3a8b2c1fd986a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:20:08.055281 14716 raft_consensus.cc:482] T 00000000000000000000000000000000 P caa10ca8dd4d468397f3a8b2c1fd986a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:20:08.055382 14716 raft_consensus.cc:3037] T 00000000000000000000000000000000 P caa10ca8dd4d468397f3a8b2c1fd986a [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:20:08.056214 14716 raft_consensus.cc:504] T 00000000000000000000000000000000 P caa10ca8dd4d468397f3a8b2c1fd986a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caa10ca8dd4d468397f3a8b2c1fd986a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37097 } }
I20260809 02:20:08.056551 14716 leader_election.cc:302] T 00000000000000000000000000000000 P caa10ca8dd4d468397f3a8b2c1fd986a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: caa10ca8dd4d468397f3a8b2c1fd986a; no voters:
I20260809 02:20:08.056906 14716 leader_election.cc:288] T 00000000000000000000000000000000 P caa10ca8dd4d468397f3a8b2c1fd986a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:20:08.057013 14721 raft_consensus.cc:2781] T 00000000000000000000000000000000 P caa10ca8dd4d468397f3a8b2c1fd986a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:20:08.057216 14721 raft_consensus.cc:686] T 00000000000000000000000000000000 P caa10ca8dd4d468397f3a8b2c1fd986a [term 1 LEADER]: Becoming Leader. State: Replica: caa10ca8dd4d468397f3a8b2c1fd986a, State: Running, Role: LEADER
I20260809 02:20:08.057559 14721 consensus_queue.cc:237] T 00000000000000000000000000000000 P caa10ca8dd4d468397f3a8b2c1fd986a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caa10ca8dd4d468397f3a8b2c1fd986a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37097 } }
I20260809 02:20:08.058241 14716 sys_catalog.cc:564] T 00000000000000000000000000000000 P caa10ca8dd4d468397f3a8b2c1fd986a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:20:08.059131 14722 sys_catalog.cc:455] T 00000000000000000000000000000000 P caa10ca8dd4d468397f3a8b2c1fd986a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "caa10ca8dd4d468397f3a8b2c1fd986a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caa10ca8dd4d468397f3a8b2c1fd986a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37097 } } }
I20260809 02:20:08.059252 14722 sys_catalog.cc:458] T 00000000000000000000000000000000 P caa10ca8dd4d468397f3a8b2c1fd986a [sys.catalog]: This master's current role is: LEADER
I20260809 02:20:08.059620 14723 sys_catalog.cc:455] T 00000000000000000000000000000000 P caa10ca8dd4d468397f3a8b2c1fd986a [sys.catalog]: SysCatalogTable state changed. Reason: New leader caa10ca8dd4d468397f3a8b2c1fd986a. Latest consensus state: current_term: 1 leader_uuid: "caa10ca8dd4d468397f3a8b2c1fd986a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caa10ca8dd4d468397f3a8b2c1fd986a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37097 } } }
I20260809 02:20:08.059759 14723 sys_catalog.cc:458] T 00000000000000000000000000000000 P caa10ca8dd4d468397f3a8b2c1fd986a [sys.catalog]: This master's current role is: LEADER
I20260809 02:20:08.060678 14719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:08.046230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44849 (local address 127.0.0.1:37097)
0809 02:20:08.046644 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:08.046659 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:08.046693 (+ 34us) server_negotiation.cc:407] Connection header received
0809 02:20:08.047349 (+ 656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:08.047366 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:08.047737 (+ 371us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:08.048106 (+ 369us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:08.049002 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:08.050021 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:08.051242 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:08.051708 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:08.051826 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:08.054715 (+ 2889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:20:08.054721 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:20:08.054738 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:20:08.054772 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:20:08.056739 (+ 1967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:08.057261 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:08.057267 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:08.057273 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:08.057407 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:08.057733 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:08.057736 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:08.057738 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:08.058215 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:20:08.059126 (+ 911us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:08.059316 (+ 190us) server_negotiation.cc:299] Negotiation successful
0809 02:20:08.059629 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":118,"threads_started":1}
I20260809 02:20:08.060825 14739 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:20:08.063657 15662 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:20:08.064527 14739 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:20:08.070227 14739 catalog_manager.cc:1347] Generated new cluster ID: b1a1e7b93211486a98f691485bc14872
I20260809 02:20:08.070333 14739 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:20:08.076890 14719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:08.069046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43857 (local address 127.0.0.1:37097)
0809 02:20:08.069151 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:08.069156 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:08.069433 (+ 277us) server_negotiation.cc:407] Connection header received
0809 02:20:08.069552 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:08.069556 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:08.069606 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:08.069689 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:08.070567 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:08.071097 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:08.071919 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:08.072174 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:08.072305 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:08.073706 (+ 1401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:20:08.073710 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:20:08.073713 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:20:08.073738 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:20:08.075186 (+ 1448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:08.075620 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:08.075624 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:08.075626 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:08.075710 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:08.076001 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:08.076004 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:08.076006 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:08.076181 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:20:08.076345 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:08.076567 (+ 222us) server_negotiation.cc:299] Negotiation successful
0809 02:20:08.076709 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":39}
I20260809 02:20:08.082265 14739 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:20:08.083415 14739 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:20:08.088446 14739 catalog_manager.cc:5868] T 00000000000000000000000000000000 P caa10ca8dd4d468397f3a8b2c1fd986a: Generated new TSK 0
I20260809 02:20:08.089138 14739 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:20:08.095249 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 14614
2026-08-09T02:20:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1597 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241741788589-15662-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:20:08 dist-test-slave-s11t krb5kdc[14765](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:20:08 dist-test-slave-s11t krb5kdc[14765](info): set up 2 sockets
Aug 09 02:20:08 dist-test-slave-s11t krb5kdc[14765](info): commencing operation
krb5kdc: starting...
W20260809 02:20:09.328361 15662 mini_kdc.cc:121] Time spent starting KDC: real 1.205s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Aug 09 02:20:09 dist-test-slave-s11t krb5kdc[14765](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786242009, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:20:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:20:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241741788589-15662-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241741788589-15662-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 02:20:09.449226 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241741788589-15662-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241741788589-15662-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241741788589-15662-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44663
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241741788589-15662-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44663
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:20:09.558992 14782 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:20:09.559260 14782 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:20:09.559335 14782 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:20:09.563230 14782 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:20:09.563313 14782 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:20:09.563346 14782 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:20:09.563386 14782 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:20:09.563416 14782 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:20:09.568404 14782 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241741788589-15662-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241741788589-15662-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44663
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241741788589-15662-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44663
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14782
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241741788589-15662-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:20:09.569372 14782 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:20:09.570328 14782 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:20:09.578457 14790 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:20:09.578477 14792 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:20:09.578477 14782 server_base.cc:1034] running on GCE node
W20260809 02:20:09.578497 14788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:20:09.579268 14782 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:20:09.580245 14782 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:20:09.581434 14782 hybrid_clock.cc:648] HybridClock initialized: now 1786242009581415 us; error 29 us; skew 500 ppm
Aug 09 02:20:09 dist-test-slave-s11t krb5kdc[14765](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786242009, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:20:09.584144 14782 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 02:20:09.584980 14782 webserver.cc:460] Webserver started at http://127.0.0.1:37083/ using document root <none> and password file <none>
I20260809 02:20:09.585460 14782 fs_manager.cc:359] Metadata directory not provided
I20260809 02:20:09.585515 14782 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:20:09.585665 14782 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:20:09.587507 14782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241741788589-15662-0/minicluster-data/master-0/data/instance:
uuid: "84d3dbfbcc024d0db94825dae9a2bfb4"
format_stamp: "Formatted at 2026-08-09 02:20:09 on dist-test-slave-s11t"
server_key: "9fa9c3657a6b29965875d4be65c7e71c"
server_key_iv: "2562252fb89ec16dd24e6cdbabad63e7"
server_key_version: "encryptionkey@0"
I20260809 02:20:09.587993 14782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241741788589-15662-0/minicluster-data/master-0/wal/instance:
uuid: "84d3dbfbcc024d0db94825dae9a2bfb4"
format_stamp: "Formatted at 2026-08-09 02:20:09 on dist-test-slave-s11t"
server_key: "9fa9c3657a6b29965875d4be65c7e71c"
server_key_iv: "2562252fb89ec16dd24e6cdbabad63e7"
server_key_version: "encryptionkey@0"
I20260809 02:20:09.591380 14782 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260809 02:20:09.593590 14799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:20:09.594780 14782 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 02:20:09.594901 14782 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241741788589-15662-0/minicluster-data/master-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241741788589-15662-0/minicluster-data/master-0/wal
uuid: "84d3dbfbcc024d0db94825dae9a2bfb4"
format_stamp: "Formatted at 2026-08-09 02:20:09 on dist-test-slave-s11t"
server_key: "9fa9c3657a6b29965875d4be65c7e71c"
server_key_iv: "2562252fb89ec16dd24e6cdbabad63e7"
server_key_version: "encryptionkey@0"
I20260809 02:20:09.595000 14782 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241741788589-15662-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241741788589-15662-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241741788589-15662-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:20:09.624104 14782 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:20:09.627295 14782 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:20:09.627501 14782 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:20:09.634366 14782 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44663
I20260809 02:20:09.634405 14880 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44663 every 8 connection(s)
I20260809 02:20:09.635262 14782 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
I20260809 02:20:09.637884 14881 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:20:09.643076 14881 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84d3dbfbcc024d0db94825dae9a2bfb4: Bootstrap starting.
I20260809 02:20:09.644927 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 14782
I20260809 02:20:09.645076 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241741788589-15662-0/minicluster-data/master-0/wal/instance
I20260809 02:20:09.645388 15662 external_mini_cluster.cc:1428] Setting key b583e94f504103bc725ffe944fedcd36
I20260809 02:20:09.645723 14881 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 84d3dbfbcc024d0db94825dae9a2bfb4: Neither blocks nor log segments found. Creating new log.
I20260809 02:20:09.646613 14881 log.cc:824] T 00000000000000000000000000000000 P 84d3dbfbcc024d0db94825dae9a2bfb4: Log is configured to *not* fsync() on all Append() calls
I20260809 02:20:09.648461 14881 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84d3dbfbcc024d0db94825dae9a2bfb4: No bootstrap required, opened a new log
I20260809 02:20:09.651798 14881 raft_consensus.cc:348] T 00000000000000000000000000000000 P 84d3dbfbcc024d0db94825dae9a2bfb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84d3dbfbcc024d0db94825dae9a2bfb4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44663 } }
I20260809 02:20:09.652035 14881 raft_consensus.cc:374] T 00000000000000000000000000000000 P 84d3dbfbcc024d0db94825dae9a2bfb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:20:09.652128 14881 raft_consensus.cc:729] T 00000000000000000000000000000000 P 84d3dbfbcc024d0db94825dae9a2bfb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84d3dbfbcc024d0db94825dae9a2bfb4, State: Initialized, Role: FOLLOWER
I20260809 02:20:09.652597 14881 consensus_queue.cc:260] T 00000000000000000000000000000000 P 84d3dbfbcc024d0db94825dae9a2bfb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84d3dbfbcc024d0db94825dae9a2bfb4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44663 } }
I20260809 02:20:09.652765 14881 raft_consensus.cc:388] T 00000000000000000000000000000000 P 84d3dbfbcc024d0db94825dae9a2bfb4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:20:09.652868 14881 raft_consensus.cc:482] T 00000000000000000000000000000000 P 84d3dbfbcc024d0db94825dae9a2bfb4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:20:09.652973 14881 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 84d3dbfbcc024d0db94825dae9a2bfb4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:20:09.653857 14881 raft_consensus.cc:504] T 00000000000000000000000000000000 P 84d3dbfbcc024d0db94825dae9a2bfb4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84d3dbfbcc024d0db94825dae9a2bfb4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44663 } }
I20260809 02:20:09.653894 14884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:09.646553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36755 (local address 127.0.0.1:44663)
0809 02:20:09.647007 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:09.647026 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:09.647070 (+ 44us) server_negotiation.cc:407] Connection header received
0809 02:20:09.647795 (+ 725us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:09.647813 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:09.648129 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:09.648451 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:09.648883 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:09.649793 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:09.650701 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:09.651184 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:09.651370 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:09.652207 (+ 837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:20:09.652215 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:20:09.652234 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:20:09.652630 (+ 396us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:20:09.652731 (+ 101us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:20:09.652768 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:20:09.652862 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:09.652977 (+ 115us) server_negotiation.cc:299] Negotiation successful
0809 02:20:09.653187 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":131,"threads_started":1}
I20260809 02:20:09.654235 14881 leader_election.cc:302] T 00000000000000000000000000000000 P 84d3dbfbcc024d0db94825dae9a2bfb4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 84d3dbfbcc024d0db94825dae9a2bfb4; no voters:
I20260809 02:20:09.654589 14881 leader_election.cc:288] T 00000000000000000000000000000000 P 84d3dbfbcc024d0db94825dae9a2bfb4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:20:09.654691 14886 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 84d3dbfbcc024d0db94825dae9a2bfb4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:20:09.654959 14886 raft_consensus.cc:686] T 00000000000000000000000000000000 P 84d3dbfbcc024d0db94825dae9a2bfb4 [term 1 LEADER]: Becoming Leader. State: Replica: 84d3dbfbcc024d0db94825dae9a2bfb4, State: Running, Role: LEADER
I20260809 02:20:09.655284 14886 consensus_queue.cc:237] T 00000000000000000000000000000000 P 84d3dbfbcc024d0db94825dae9a2bfb4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84d3dbfbcc024d0db94825dae9a2bfb4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44663 } }
I20260809 02:20:09.655776 14881 sys_catalog.cc:564] T 00000000000000000000000000000000 P 84d3dbfbcc024d0db94825dae9a2bfb4 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:20:09.656452 14887 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84d3dbfbcc024d0db94825dae9a2bfb4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 84d3dbfbcc024d0db94825dae9a2bfb4. Latest consensus state: current_term: 1 leader_uuid: "84d3dbfbcc024d0db94825dae9a2bfb4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84d3dbfbcc024d0db94825dae9a2bfb4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44663 } } }
I20260809 02:20:09.656625 14887 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84d3dbfbcc024d0db94825dae9a2bfb4 [sys.catalog]: This master's current role is: LEADER
I20260809 02:20:09.656935 14888 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84d3dbfbcc024d0db94825dae9a2bfb4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "84d3dbfbcc024d0db94825dae9a2bfb4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84d3dbfbcc024d0db94825dae9a2bfb4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44663 } } }
I20260809 02:20:09.657068 14888 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84d3dbfbcc024d0db94825dae9a2bfb4 [sys.catalog]: This master's current role is: LEADER
I20260809 02:20:09.657121 15662 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260809 02:20:09.661028 14909 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 84d3dbfbcc024d0db94825dae9a2bfb4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:20:09.661111 14909 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:20:09.661221 14895 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:20:09.662079 14895 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:20:09.663692 14884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:09.659636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34199 (local address 127.0.0.1:44663)
0809 02:20:09.659726 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:09.659730 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:09.659742 (+ 12us) server_negotiation.cc:407] Connection header received
0809 02:20:09.659883 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:09.659886 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:09.659927 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:09.660019 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:09.660271 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:09.660863 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:09.661924 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:09.662265 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:09.662390 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:09.663218 (+ 828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:20:09.663223 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:20:09.663229 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:20:09.663242 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:20:09.663270 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:20:09.663283 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:20:09.663374 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:09.663473 (+ 99us) server_negotiation.cc:299] Negotiation successful
0809 02:20:09.663557 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 02:20:09.668409 14895 catalog_manager.cc:1347] Generated new cluster ID: 329d3089c1fe49f788bf99e8bf8e2eae
I20260809 02:20:09.668470 14895 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:20:09.687391 14895 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:20:09.688263 14895 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:20:09.692312 14895 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 84d3dbfbcc024d0db94825dae9a2bfb4: Generated new TSK 0
I20260809 02:20:09.692975 14895 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:20:09.701005 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 14782
2026-08-09T02:20:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1605 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241741788589-15662-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:20:09 dist-test-slave-s11t krb5kdc[14923](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:20:09 dist-test-slave-s11t krb5kdc[14923](info): set up 2 sockets
Aug 09 02:20:09 dist-test-slave-s11t krb5kdc[14923](info): commencing operation
krb5kdc: starting...
W20260809 02:20:10.930949 15662 mini_kdc.cc:121] Time spent starting KDC: real 1.203s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Aug 09 02:20:11 dist-test-slave-s11t krb5kdc[14923](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786242011, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:20:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:20:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241741788589-15662-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241741788589-15662-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 02:20:11.060130 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241741788589-15662-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241741788589-15662-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241741788589-15662-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40937
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241741788589-15662-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40937
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:20:11.170307 14943 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:20:11.170601 14943 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:20:11.170675 14943 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:20:11.174604 14943 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:20:11.174690 14943 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:20:11.174729 14943 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:20:11.174793 14943 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:20:11.174832 14943 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:20:11.179920 14943 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241741788589-15662-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241741788589-15662-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40937
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241741788589-15662-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40937
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14943
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241741788589-15662-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:20:11.180938 14943 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:20:11.181926 14943 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:20:11.190425 14950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:20:11.190438 14948 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:20:11.190464 14953 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:20:11.190490 14943 server_base.cc:1034] running on GCE node
I20260809 02:20:11.191188 14943 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:20:11.192245 14943 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:20:11.193436 14943 hybrid_clock.cc:648] HybridClock initialized: now 1786242011193410 us; error 39 us; skew 500 ppm
Aug 09 02:20:11 dist-test-slave-s11t krb5kdc[14923](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786242011, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:20:11.196108 14943 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 02:20:11.197001 14943 webserver.cc:460] Webserver started at http://127.0.0.1:34817/ using document root <none> and password file <none>
I20260809 02:20:11.197535 14943 fs_manager.cc:359] Metadata directory not provided
I20260809 02:20:11.197620 14943 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:20:11.197844 14943 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:20:11.199795 14943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241741788589-15662-0/minicluster-data/master-0/data/instance:
uuid: "eb36c66ce14e4301a6f305e3040ee472"
format_stamp: "Formatted at 2026-08-09 02:20:11 on dist-test-slave-s11t"
server_key: "8f7d127c400767d3496d2b6e69558b97"
server_key_iv: "02651c1a36246c1d5efb0923f7ff7707"
server_key_version: "encryptionkey@0"
I20260809 02:20:11.200348 14943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241741788589-15662-0/minicluster-data/master-0/wal/instance:
uuid: "eb36c66ce14e4301a6f305e3040ee472"
format_stamp: "Formatted at 2026-08-09 02:20:11 on dist-test-slave-s11t"
server_key: "8f7d127c400767d3496d2b6e69558b97"
server_key_iv: "02651c1a36246c1d5efb0923f7ff7707"
server_key_version: "encryptionkey@0"
I20260809 02:20:11.203893 14943 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260809 02:20:11.206172 14963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:20:11.207275 14943 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 02:20:11.207413 14943 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241741788589-15662-0/minicluster-data/master-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241741788589-15662-0/minicluster-data/master-0/wal
uuid: "eb36c66ce14e4301a6f305e3040ee472"
format_stamp: "Formatted at 2026-08-09 02:20:11 on dist-test-slave-s11t"
server_key: "8f7d127c400767d3496d2b6e69558b97"
server_key_iv: "02651c1a36246c1d5efb0923f7ff7707"
server_key_version: "encryptionkey@0"
I20260809 02:20:11.207515 14943 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241741788589-15662-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241741788589-15662-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241741788589-15662-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:20:11.211896 14943 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:20:11.214591 14943 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:20:11.214769 14943 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:20:11.226111 14943 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40937
I20260809 02:20:11.226383 15038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40937 every 8 connection(s)
I20260809 02:20:11.227371 14943 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
I20260809 02:20:11.230190 15039 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:20:11.235590 15039 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb36c66ce14e4301a6f305e3040ee472: Bootstrap starting.
I20260809 02:20:11.236078 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 14943
I20260809 02:20:11.236212 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241741788589-15662-0/minicluster-data/master-0/wal/instance
I20260809 02:20:11.236490 15662 external_mini_cluster.cc:1428] Setting key a55738566a2d4df963470144437fa1bd
I20260809 02:20:11.238234 15039 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eb36c66ce14e4301a6f305e3040ee472: Neither blocks nor log segments found. Creating new log.
I20260809 02:20:11.238965 15039 log.cc:824] T 00000000000000000000000000000000 P eb36c66ce14e4301a6f305e3040ee472: Log is configured to *not* fsync() on all Append() calls
I20260809 02:20:11.241206 15039 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb36c66ce14e4301a6f305e3040ee472: No bootstrap required, opened a new log
I20260809 02:20:11.241478 15043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:11.237740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37859 (local address 127.0.0.1:40937)
0809 02:20:11.238147 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:11.238161 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:11.238194 (+ 33us) server_negotiation.cc:407] Connection header received
0809 02:20:11.238820 (+ 626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:11.238836 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:11.239141 (+ 305us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:11.239493 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:11.240268 (+ 775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:20:11.240272 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:20:11.240284 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:20:11.240294 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:20:11.240348 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:20:11.240375 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:20:11.240495 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:11.240614 (+ 119us) server_negotiation.cc:299] Negotiation successful
0809 02:20:11.240787 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":130,"threads_started":1}
I20260809 02:20:11.244102 15662 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:20:11.245065 15039 raft_consensus.cc:348] T 00000000000000000000000000000000 P eb36c66ce14e4301a6f305e3040ee472 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb36c66ce14e4301a6f305e3040ee472" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40937 } }
I20260809 02:20:11.245265 15039 raft_consensus.cc:374] T 00000000000000000000000000000000 P eb36c66ce14e4301a6f305e3040ee472 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:20:11.245405 15039 raft_consensus.cc:729] T 00000000000000000000000000000000 P eb36c66ce14e4301a6f305e3040ee472 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb36c66ce14e4301a6f305e3040ee472, State: Initialized, Role: FOLLOWER
I20260809 02:20:11.245859 15039 consensus_queue.cc:260] T 00000000000000000000000000000000 P eb36c66ce14e4301a6f305e3040ee472 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb36c66ce14e4301a6f305e3040ee472" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40937 } }
I20260809 02:20:11.246029 15039 raft_consensus.cc:388] T 00000000000000000000000000000000 P eb36c66ce14e4301a6f305e3040ee472 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:20:11.246104 15039 raft_consensus.cc:482] T 00000000000000000000000000000000 P eb36c66ce14e4301a6f305e3040ee472 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:20:11.246227 15039 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eb36c66ce14e4301a6f305e3040ee472 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:20:11.247401 15039 raft_consensus.cc:504] T 00000000000000000000000000000000 P eb36c66ce14e4301a6f305e3040ee472 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb36c66ce14e4301a6f305e3040ee472" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40937 } }
I20260809 02:20:11.247776 15039 leader_election.cc:302] T 00000000000000000000000000000000 P eb36c66ce14e4301a6f305e3040ee472 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eb36c66ce14e4301a6f305e3040ee472; no voters:
I20260809 02:20:11.247900 15043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:11.246305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42485 (local address 127.0.0.1:40937)
0809 02:20:11.246418 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:11.246421 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:11.246608 (+ 187us) server_negotiation.cc:407] Connection header received
0809 02:20:11.246733 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:11.246735 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:11.246759 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:11.246819 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:11.247474 (+ 655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:20:11.247477 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:20:11.247479 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:20:11.247488 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:20:11.247510 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:20:11.247520 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:20:11.247631 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:11.247714 (+ 83us) server_negotiation.cc:299] Negotiation successful
0809 02:20:11.247741 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 02:20:11.248142 15039 leader_election.cc:288] T 00000000000000000000000000000000 P eb36c66ce14e4301a6f305e3040ee472 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:20:11.248203 15052 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eb36c66ce14e4301a6f305e3040ee472 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:20:11.248438 15052 raft_consensus.cc:686] T 00000000000000000000000000000000 P eb36c66ce14e4301a6f305e3040ee472 [term 1 LEADER]: Becoming Leader. State: Replica: eb36c66ce14e4301a6f305e3040ee472, State: Running, Role: LEADER
I20260809 02:20:11.248798 15052 consensus_queue.cc:237] T 00000000000000000000000000000000 P eb36c66ce14e4301a6f305e3040ee472 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb36c66ce14e4301a6f305e3040ee472" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40937 } }
I20260809 02:20:11.249446 15039 sys_catalog.cc:564] T 00000000000000000000000000000000 P eb36c66ce14e4301a6f305e3040ee472 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:20:11.250524 15054 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb36c66ce14e4301a6f305e3040ee472 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eb36c66ce14e4301a6f305e3040ee472" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb36c66ce14e4301a6f305e3040ee472" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40937 } } }
I20260809 02:20:11.250564 15053 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb36c66ce14e4301a6f305e3040ee472 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eb36c66ce14e4301a6f305e3040ee472. Latest consensus state: current_term: 1 leader_uuid: "eb36c66ce14e4301a6f305e3040ee472" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb36c66ce14e4301a6f305e3040ee472" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40937 } } }
I20260809 02:20:11.250644 15054 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb36c66ce14e4301a6f305e3040ee472 [sys.catalog]: This master's current role is: LEADER
I20260809 02:20:11.250669 15053 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb36c66ce14e4301a6f305e3040ee472 [sys.catalog]: This master's current role is: LEADER
I20260809 02:20:11.251003 15064 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:20:11.254125 15064 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:20:11.259400 15064 catalog_manager.cc:1347] Generated new cluster ID: 7f167d1d701049c1bc40b30a1204402d
I20260809 02:20:11.259466 15064 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:20:11.279016 15064 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:20:11.279330 15064 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:20:11.287783 15064 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eb36c66ce14e4301a6f305e3040ee472: Generated new TSK 0
I20260809 02:20:11.288508 15064 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:20:11.292908 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 14943
2026-08-09T02:20:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1589 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241741788589-15662-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:20:11 dist-test-slave-s11t krb5kdc[15088](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:20:11 dist-test-slave-s11t krb5kdc[15088](info): set up 2 sockets
Aug 09 02:20:11 dist-test-slave-s11t krb5kdc[15088](info): commencing operation
krb5kdc: starting...
W20260809 02:20:12.528625 15662 mini_kdc.cc:121] Time spent starting KDC: real 1.210s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Aug 09 02:20:12 dist-test-slave-s11t krb5kdc[15088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786242012, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:20:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:20:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241741788589-15662-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241741788589-15662-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 02:20:12.649789 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241741788589-15662-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241741788589-15662-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241741788589-15662-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33707
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241741788589-15662-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33707
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:20:12.757396 15113 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:20:12.757658 15113 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:20:12.757730 15113 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:20:12.761773 15113 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:20:12.761842 15113 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:20:12.761888 15113 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:20:12.761922 15113 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:20:12.761957 15113 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:20:12.766995 15113 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241741788589-15662-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241741788589-15662-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33707
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241741788589-15662-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33707
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15113
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241741788589-15662-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:20:12.767972 15113 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:20:12.768954 15113 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:20:12.777072 15124 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:20:12.777097 15121 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:20:12.777177 15120 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:20:12.777478 15113 server_base.cc:1034] running on GCE node
I20260809 02:20:12.778013 15113 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:20:12.779011 15113 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:20:12.780198 15113 hybrid_clock.cc:648] HybridClock initialized: now 1786242012780181 us; error 29 us; skew 500 ppm
Aug 09 02:20:12 dist-test-slave-s11t krb5kdc[15088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786242012, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:20:12.782971 15113 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 02:20:12.783839 15113 webserver.cc:460] Webserver started at http://127.0.0.1:43239/ using document root <none> and password file <none>
I20260809 02:20:12.784377 15113 fs_manager.cc:359] Metadata directory not provided
I20260809 02:20:12.784469 15113 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:20:12.784754 15113 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:20:12.786692 15113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241741788589-15662-0/minicluster-data/master-0/data/instance:
uuid: "6c6151b8f07d468aa1304dbda086f581"
format_stamp: "Formatted at 2026-08-09 02:20:12 on dist-test-slave-s11t"
server_key: "e507649a65d821fa6f60dd4d6680f6a4"
server_key_iv: "951b805c99938837f00e780e7d90d7af"
server_key_version: "encryptionkey@0"
I20260809 02:20:12.787251 15113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241741788589-15662-0/minicluster-data/master-0/wal/instance:
uuid: "6c6151b8f07d468aa1304dbda086f581"
format_stamp: "Formatted at 2026-08-09 02:20:12 on dist-test-slave-s11t"
server_key: "e507649a65d821fa6f60dd4d6680f6a4"
server_key_iv: "951b805c99938837f00e780e7d90d7af"
server_key_version: "encryptionkey@0"
I20260809 02:20:12.790802 15113 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.005s
I20260809 02:20:12.793087 15135 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:20:12.794201 15113 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 02:20:12.794339 15113 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241741788589-15662-0/minicluster-data/master-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241741788589-15662-0/minicluster-data/master-0/wal
uuid: "6c6151b8f07d468aa1304dbda086f581"
format_stamp: "Formatted at 2026-08-09 02:20:12 on dist-test-slave-s11t"
server_key: "e507649a65d821fa6f60dd4d6680f6a4"
server_key_iv: "951b805c99938837f00e780e7d90d7af"
server_key_version: "encryptionkey@0"
I20260809 02:20:12.794446 15113 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241741788589-15662-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241741788589-15662-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241741788589-15662-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:20:12.826696 15113 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:20:12.829999 15113 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:20:12.830205 15113 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:20:12.837368 15113 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33707
I20260809 02:20:12.837412 15214 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33707 every 8 connection(s)
I20260809 02:20:12.838289 15113 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
I20260809 02:20:12.840992 15216 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:20:12.845824 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 15113
I20260809 02:20:12.845976 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241741788589-15662-0/minicluster-data/master-0/wal/instance
I20260809 02:20:12.846333 15662 external_mini_cluster.cc:1428] Setting key cf2d4eb04ff20bd0454af7674caadc8e
I20260809 02:20:12.847018 15216 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c6151b8f07d468aa1304dbda086f581: Bootstrap starting.
I20260809 02:20:12.849488 15216 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6c6151b8f07d468aa1304dbda086f581: Neither blocks nor log segments found. Creating new log.
I20260809 02:20:12.850227 15216 log.cc:824] T 00000000000000000000000000000000 P 6c6151b8f07d468aa1304dbda086f581: Log is configured to *not* fsync() on all Append() calls
I20260809 02:20:12.852023 15216 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c6151b8f07d468aa1304dbda086f581: No bootstrap required, opened a new log
Aug 09 02:20:12 dist-test-slave-s11t krb5kdc[15088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786242012, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 02:20:12.855923 15216 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6c6151b8f07d468aa1304dbda086f581 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c6151b8f07d468aa1304dbda086f581" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33707 } }
I20260809 02:20:12.856127 15216 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6c6151b8f07d468aa1304dbda086f581 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:20:12.856248 15216 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6c6151b8f07d468aa1304dbda086f581 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c6151b8f07d468aa1304dbda086f581, State: Initialized, Role: FOLLOWER
I20260809 02:20:12.856664 15216 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6c6151b8f07d468aa1304dbda086f581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c6151b8f07d468aa1304dbda086f581" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33707 } }
I20260809 02:20:12.856859 15216 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6c6151b8f07d468aa1304dbda086f581 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:20:12.856933 15216 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6c6151b8f07d468aa1304dbda086f581 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:20:12.857046 15216 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6c6151b8f07d468aa1304dbda086f581 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:20:12.857895 15216 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6c6151b8f07d468aa1304dbda086f581 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c6151b8f07d468aa1304dbda086f581" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33707 } }
I20260809 02:20:12.858274 15216 leader_election.cc:302] T 00000000000000000000000000000000 P 6c6151b8f07d468aa1304dbda086f581 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6c6151b8f07d468aa1304dbda086f581; no voters:
I20260809 02:20:12.858618 15216 leader_election.cc:288] T 00000000000000000000000000000000 P 6c6151b8f07d468aa1304dbda086f581 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:20:12.858711 15224 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6c6151b8f07d468aa1304dbda086f581 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:20:12.858965 15224 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6c6151b8f07d468aa1304dbda086f581 [term 1 LEADER]: Becoming Leader. State: Replica: 6c6151b8f07d468aa1304dbda086f581, State: Running, Role: LEADER
I20260809 02:20:12.859308 15224 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6c6151b8f07d468aa1304dbda086f581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c6151b8f07d468aa1304dbda086f581" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33707 } }
I20260809 02:20:12.859932 15216 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6c6151b8f07d468aa1304dbda086f581 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:20:12.861569 15220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:12.847777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.76:55191 (local address 127.0.0.1:33707)
0809 02:20:12.848191 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:12.848206 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:12.848239 (+ 33us) server_negotiation.cc:407] Connection header received
0809 02:20:12.848925 (+ 686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:12.848942 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:12.849266 (+ 324us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:12.849594 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:12.850430 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:12.851358 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:12.852261 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:12.852780 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:12.852957 (+ 177us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:12.855739 (+ 2782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:20:12.855744 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:20:12.855756 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:20:12.855780 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:20:12.857730 (+ 1950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:12.858295 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:12.858300 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:12.858304 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:12.858464 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:12.858870 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:12.858874 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:12.858878 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:12.859377 (+ 499us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:20:12.859646 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:12.860290 (+ 644us) server_negotiation.cc:299] Negotiation successful
0809 02:20:12.860511 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":135,"threads_started":1}
I20260809 02:20:12.862866 15227 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c6151b8f07d468aa1304dbda086f581 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c6151b8f07d468aa1304dbda086f581. Latest consensus state: current_term: 1 leader_uuid: "6c6151b8f07d468aa1304dbda086f581" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c6151b8f07d468aa1304dbda086f581" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33707 } } }
I20260809 02:20:12.862911 15226 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c6151b8f07d468aa1304dbda086f581 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6c6151b8f07d468aa1304dbda086f581" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c6151b8f07d468aa1304dbda086f581" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33707 } } }
I20260809 02:20:12.863045 15227 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c6151b8f07d468aa1304dbda086f581 [sys.catalog]: This master's current role is: LEADER
I20260809 02:20:12.863044 15226 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c6151b8f07d468aa1304dbda086f581 [sys.catalog]: This master's current role is: LEADER
I20260809 02:20:12.865105 15662 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260809 02:20:12.866662 15243 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6c6151b8f07d468aa1304dbda086f581: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:20:12.866741 15243 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:20:12.866820 15246 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:20:12.867707 15246 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:20:12.874068 15246 catalog_manager.cc:1347] Generated new cluster ID: 347c94b1cc394e2c91d3df1b78b6942d
I20260809 02:20:12.874127 15246 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:20:12.877519 15220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:12.867122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.76:42797 (local address 127.0.0.1:33707)
0809 02:20:12.867246 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:12.867250 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:12.869056 (+ 1806us) server_negotiation.cc:407] Connection header received
0809 02:20:12.869217 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:12.869220 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:12.869260 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:12.869342 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:12.870145 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:12.871017 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:12.871978 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:12.872354 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:12.872467 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:12.873880 (+ 1413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:20:12.873884 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:20:12.873890 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:20:12.873924 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:20:12.875641 (+ 1717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:12.876203 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:12.876206 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:12.876208 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:12.876284 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:12.876618 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:12.876621 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:12.876623 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:12.876846 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:20:12.876992 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:12.877290 (+ 298us) server_negotiation.cc:299] Negotiation successful
0809 02:20:12.877378 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 02:20:12.891315 15246 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:20:12.892521 15246 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:20:12.897987 15246 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6c6151b8f07d468aa1304dbda086f581: Generated new TSK 0
I20260809 02:20:12.898711 15246 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:20:12.903723 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 15113
W20260809 02:20:12.919467 15084 connection.cc:535] client connection to 127.0.0.1:33707 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33707): Connection reset by peer (error 104)
2026-08-09T02:20:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1615 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241741788589-15662-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:20:12 dist-test-slave-s11t krb5kdc[15262](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:20:12 dist-test-slave-s11t krb5kdc[15262](info): set up 2 sockets
Aug 09 02:20:12 dist-test-slave-s11t krb5kdc[15262](info): commencing operation
krb5kdc: starting...
W20260809 02:20:14.131865 15662 mini_kdc.cc:121] Time spent starting KDC: real 1.198s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Aug 09 02:20:14 dist-test-slave-s11t krb5kdc[15262](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786242014, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:20:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:20:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241741788589-15662-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241741788589-15662-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 02:20:14.253129 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241741788589-15662-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241741788589-15662-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241741788589-15662-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38743
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241741788589-15662-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38743
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:20:14.362071 15285 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:20:14.362299 15285 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:20:14.362339 15285 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:20:14.366355 15285 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:20:14.366406 15285 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:20:14.366425 15285 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:20:14.366441 15285 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:20:14.366456 15285 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:20:14.371465 15285 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241741788589-15662-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241741788589-15662-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38743
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241741788589-15662-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38743
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15285
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241741788589-15662-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:20:14.372329 15285 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:20:14.373245 15285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:20:14.381541 15293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:20:14.381587 15296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:20:14.381542 15294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:20:14.381880 15285 server_base.cc:1034] running on GCE node
I20260809 02:20:14.382532 15285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:20:14.383574 15285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:20:14.384790 15285 hybrid_clock.cc:648] HybridClock initialized: now 1786242014384752 us; error 50 us; skew 500 ppm
Aug 09 02:20:14 dist-test-slave-s11t krb5kdc[15262](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786242014, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:20:14.390434 15285 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 02:20:14.391340 15285 webserver.cc:460] Webserver started at http://127.0.0.1:39769/ using document root <none> and password file <none>
I20260809 02:20:14.391909 15285 fs_manager.cc:359] Metadata directory not provided
I20260809 02:20:14.392004 15285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:20:14.392232 15285 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:20:14.394256 15285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241741788589-15662-0/minicluster-data/master-0/data/instance:
uuid: "f069c4547cf44dcd8faaa4b4ae8a275c"
format_stamp: "Formatted at 2026-08-09 02:20:14 on dist-test-slave-s11t"
server_key: "3df6e87a3e7564c7ebd4266d3686b539"
server_key_iv: "3bb7e339877c993b5e69777a3a9ebdc5"
server_key_version: "encryptionkey@0"
I20260809 02:20:14.394814 15285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241741788589-15662-0/minicluster-data/master-0/wal/instance:
uuid: "f069c4547cf44dcd8faaa4b4ae8a275c"
format_stamp: "Formatted at 2026-08-09 02:20:14 on dist-test-slave-s11t"
server_key: "3df6e87a3e7564c7ebd4266d3686b539"
server_key_iv: "3bb7e339877c993b5e69777a3a9ebdc5"
server_key_version: "encryptionkey@0"
I20260809 02:20:14.398427 15285 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.001s
I20260809 02:20:14.400748 15309 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:20:14.401979 15285 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 02:20:14.402125 15285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241741788589-15662-0/minicluster-data/master-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241741788589-15662-0/minicluster-data/master-0/wal
uuid: "f069c4547cf44dcd8faaa4b4ae8a275c"
format_stamp: "Formatted at 2026-08-09 02:20:14 on dist-test-slave-s11t"
server_key: "3df6e87a3e7564c7ebd4266d3686b539"
server_key_iv: "3bb7e339877c993b5e69777a3a9ebdc5"
server_key_version: "encryptionkey@0"
I20260809 02:20:14.402232 15285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241741788589-15662-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241741788589-15662-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241741788589-15662-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:20:14.415380 15285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:20:14.418087 15285 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:20:14.418282 15285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:20:14.425257 15285 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38743
I20260809 02:20:14.425292 15394 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38743 every 8 connection(s)
I20260809 02:20:14.426127 15285 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
I20260809 02:20:14.428674 15396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:20:14.428974 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 15285
I20260809 02:20:14.429136 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241741788589-15662-0/minicluster-data/master-0/wal/instance
I20260809 02:20:14.429430 15662 external_mini_cluster.cc:1428] Setting key 17dcc250145f4eedc1fe0c471cac9f13
I20260809 02:20:14.434929 15396 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f069c4547cf44dcd8faaa4b4ae8a275c: Bootstrap starting.
I20260809 02:20:14.437232 15396 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f069c4547cf44dcd8faaa4b4ae8a275c: Neither blocks nor log segments found. Creating new log.
I20260809 02:20:14.437956 15396 log.cc:824] T 00000000000000000000000000000000 P f069c4547cf44dcd8faaa4b4ae8a275c: Log is configured to *not* fsync() on all Append() calls
I20260809 02:20:14.437996 15402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:14.430749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.76:40585 (local address 127.0.0.1:38743)
0809 02:20:14.431171 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:14.431185 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:14.431217 (+ 32us) server_negotiation.cc:407] Connection header received
0809 02:20:14.431867 (+ 650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:14.431882 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:14.432198 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:14.432514 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:14.432946 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:14.433916 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:14.434808 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:14.435321 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:14.435486 (+ 165us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:14.436378 (+ 892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:20:14.436383 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:20:14.436397 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:20:14.436801 (+ 404us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:20:14.436853 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:20:14.436875 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:20:14.437016 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:14.437199 (+ 183us) server_negotiation.cc:299] Negotiation successful
0809 02:20:14.437349 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":135,"threads_started":1}
I20260809 02:20:14.439774 15396 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f069c4547cf44dcd8faaa4b4ae8a275c: No bootstrap required, opened a new log
I20260809 02:20:14.440716 15662 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:20:14.443547 15396 raft_consensus.cc:348] T 00000000000000000000000000000000 P f069c4547cf44dcd8faaa4b4ae8a275c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f069c4547cf44dcd8faaa4b4ae8a275c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38743 } }
I20260809 02:20:14.443740 15396 raft_consensus.cc:374] T 00000000000000000000000000000000 P f069c4547cf44dcd8faaa4b4ae8a275c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:20:14.443822 15396 raft_consensus.cc:729] T 00000000000000000000000000000000 P f069c4547cf44dcd8faaa4b4ae8a275c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f069c4547cf44dcd8faaa4b4ae8a275c, State: Initialized, Role: FOLLOWER
I20260809 02:20:14.444295 15396 consensus_queue.cc:260] T 00000000000000000000000000000000 P f069c4547cf44dcd8faaa4b4ae8a275c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f069c4547cf44dcd8faaa4b4ae8a275c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38743 } }
I20260809 02:20:14.444454 15396 raft_consensus.cc:388] T 00000000000000000000000000000000 P f069c4547cf44dcd8faaa4b4ae8a275c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:20:14.444521 15396 raft_consensus.cc:482] T 00000000000000000000000000000000 P f069c4547cf44dcd8faaa4b4ae8a275c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:20:14.444609 15396 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f069c4547cf44dcd8faaa4b4ae8a275c [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:20:14.445621 15396 raft_consensus.cc:504] T 00000000000000000000000000000000 P f069c4547cf44dcd8faaa4b4ae8a275c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f069c4547cf44dcd8faaa4b4ae8a275c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38743 } }
I20260809 02:20:14.445968 15396 leader_election.cc:302] T 00000000000000000000000000000000 P f069c4547cf44dcd8faaa4b4ae8a275c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f069c4547cf44dcd8faaa4b4ae8a275c; no voters:
I20260809 02:20:14.446334 15396 leader_election.cc:288] T 00000000000000000000000000000000 P f069c4547cf44dcd8faaa4b4ae8a275c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:20:14.446421 15412 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f069c4547cf44dcd8faaa4b4ae8a275c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:20:14.446647 15412 raft_consensus.cc:686] T 00000000000000000000000000000000 P f069c4547cf44dcd8faaa4b4ae8a275c [term 1 LEADER]: Becoming Leader. State: Replica: f069c4547cf44dcd8faaa4b4ae8a275c, State: Running, Role: LEADER
I20260809 02:20:14.447048 15402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:14.442566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.76:45485 (local address 127.0.0.1:38743)
0809 02:20:14.442671 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:14.442675 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:14.442861 (+ 186us) server_negotiation.cc:407] Connection header received
0809 02:20:14.443017 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:14.443019 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:14.443064 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:14.443143 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:14.443398 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:14.444016 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:14.445176 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:14.445418 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:14.445496 (+ 78us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:14.446314 (+ 818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:20:14.446319 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:20:14.446321 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:20:14.446335 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:20:14.446362 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:20:14.446376 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:20:14.446485 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:14.446897 (+ 412us) server_negotiation.cc:299] Negotiation successful
0809 02:20:14.446929 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 02:20:14.447016 15412 consensus_queue.cc:237] T 00000000000000000000000000000000 P f069c4547cf44dcd8faaa4b4ae8a275c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f069c4547cf44dcd8faaa4b4ae8a275c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38743 } }
I20260809 02:20:14.447530 15396 sys_catalog.cc:564] T 00000000000000000000000000000000 P f069c4547cf44dcd8faaa4b4ae8a275c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:20:14.448767 15413 sys_catalog.cc:455] T 00000000000000000000000000000000 P f069c4547cf44dcd8faaa4b4ae8a275c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f069c4547cf44dcd8faaa4b4ae8a275c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f069c4547cf44dcd8faaa4b4ae8a275c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38743 } } }
I20260809 02:20:14.448824 15414 sys_catalog.cc:455] T 00000000000000000000000000000000 P f069c4547cf44dcd8faaa4b4ae8a275c [sys.catalog]: SysCatalogTable state changed. Reason: New leader f069c4547cf44dcd8faaa4b4ae8a275c. Latest consensus state: current_term: 1 leader_uuid: "f069c4547cf44dcd8faaa4b4ae8a275c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f069c4547cf44dcd8faaa4b4ae8a275c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38743 } } }
I20260809 02:20:14.448910 15413 sys_catalog.cc:458] T 00000000000000000000000000000000 P f069c4547cf44dcd8faaa4b4ae8a275c [sys.catalog]: This master's current role is: LEADER
I20260809 02:20:14.448917 15414 sys_catalog.cc:458] T 00000000000000000000000000000000 P f069c4547cf44dcd8faaa4b4ae8a275c [sys.catalog]: This master's current role is: LEADER
I20260809 02:20:14.449379 15423 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:20:14.452461 15423 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:20:14.458020 15423 catalog_manager.cc:1347] Generated new cluster ID: a01692a298b64219b81d061c172bce7d
I20260809 02:20:14.458112 15423 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:20:14.483386 15423 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:20:14.484329 15423 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:20:14.493510 15423 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f069c4547cf44dcd8faaa4b4ae8a275c: Generated new TSK 0
I20260809 02:20:14.494144 15423 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:20:14.506343 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 15285
W20260809 02:20:14.517732 15257 connection.cc:535] client connection to 127.0.0.1:38743 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38743): Connection reset by peer (error 104)
2026-08-09T02:20:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1598 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241741788589-15662-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:20:14 dist-test-slave-s11t krb5kdc[15438](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:20:14 dist-test-slave-s11t krb5kdc[15438](info): set up 2 sockets
Aug 09 02:20:14 dist-test-slave-s11t krb5kdc[15438](info): commencing operation
krb5kdc: starting...
W20260809 02:20:15.738488 15662 mini_kdc.cc:121] Time spent starting KDC: real 1.206s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Aug 09 02:20:15 dist-test-slave-s11t krb5kdc[15438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786242015, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:20:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:20:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241741788589-15662-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241741788589-15662-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 02:20:15.861127 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241741788589-15662-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241741788589-15662-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241741788589-15662-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45797
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241741788589-15662-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45797
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:20:15.971009 15458 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:20:15.971289 15458 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:20:15.971367 15458 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:20:15.975343 15458 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:20:15.975433 15458 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:20:15.975472 15458 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:20:15.975530 15458 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:20:15.975566 15458 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:20:15.980779 15458 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241741788589-15662-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241741788589-15662-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45797
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241741788589-15662-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45797
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15458
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241741788589-15662-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:20:15.981822 15458 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:20:15.982861 15458 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:20:15.991499 15467 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:20:15.991510 15468 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:20:15.991510 15471 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:20:15.991793 15458 server_base.cc:1034] running on GCE node
I20260809 02:20:15.992347 15458 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:20:15.993438 15458 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:20:15.994602 15458 hybrid_clock.cc:648] HybridClock initialized: now 1786242015994574 us; error 38 us; skew 500 ppm
Aug 09 02:20:15 dist-test-slave-s11t krb5kdc[15438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786242015, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:20:15.997555 15458 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 02:20:15.998416 15458 webserver.cc:460] Webserver started at http://127.0.0.1:38455/ using document root <none> and password file <none>
I20260809 02:20:15.998967 15458 fs_manager.cc:359] Metadata directory not provided
I20260809 02:20:15.999030 15458 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:20:15.999270 15458 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:20:16.001344 15458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241741788589-15662-0/minicluster-data/master-0/data/instance:
uuid: "521b81a530ed46deab1049ac1680b407"
format_stamp: "Formatted at 2026-08-09 02:20:15 on dist-test-slave-s11t"
server_key: "bb9b370c93150cad1233bac0f88e84ac"
server_key_iv: "b44ae010bba4732bc2252175b9e363c2"
server_key_version: "encryptionkey@0"
I20260809 02:20:16.001920 15458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241741788589-15662-0/minicluster-data/master-0/wal/instance:
uuid: "521b81a530ed46deab1049ac1680b407"
format_stamp: "Formatted at 2026-08-09 02:20:15 on dist-test-slave-s11t"
server_key: "bb9b370c93150cad1233bac0f88e84ac"
server_key_iv: "b44ae010bba4732bc2252175b9e363c2"
server_key_version: "encryptionkey@0"
I20260809 02:20:16.005637 15458 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 02:20:16.007989 15484 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:20:16.009434 15458 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 02:20:16.009560 15458 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241741788589-15662-0/minicluster-data/master-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241741788589-15662-0/minicluster-data/master-0/wal
uuid: "521b81a530ed46deab1049ac1680b407"
format_stamp: "Formatted at 2026-08-09 02:20:15 on dist-test-slave-s11t"
server_key: "bb9b370c93150cad1233bac0f88e84ac"
server_key_iv: "b44ae010bba4732bc2252175b9e363c2"
server_key_version: "encryptionkey@0"
I20260809 02:20:16.009672 15458 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241741788589-15662-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241741788589-15662-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241741788589-15662-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:20:16.014880 15458 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:20:16.017783 15458 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:20:16.017951 15458 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:20:16.025174 15458 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45797
I20260809 02:20:16.025198 15558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45797 every 8 connection(s)
I20260809 02:20:16.026110 15458 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
I20260809 02:20:16.026906 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 15458
I20260809 02:20:16.027067 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241741788589-15662-0/minicluster-data/master-0/wal/instance
I20260809 02:20:16.027379 15662 external_mini_cluster.cc:1428] Setting key 91b11d26b93f2687381990ead2a4ae86
I20260809 02:20:16.029294 15559 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:20:16.032943 15563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:16.028892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.76:48167 (local address 127.0.0.1:45797)
0809 02:20:16.029303 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:16.029323 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:16.029364 (+ 41us) server_negotiation.cc:407] Connection header received
0809 02:20:16.030164 (+ 800us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:16.030186 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:16.030529 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:16.030876 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:16.031577 (+ 701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:20:16.031582 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:20:16.031596 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:20:16.031608 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:20:16.031669 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:20:16.031697 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:20:16.031833 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:16.031905 (+ 72us) server_negotiation.cc:299] Negotiation successful
0809 02:20:16.032054 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":116,"threads_started":1}
I20260809 02:20:16.035612 15662 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:20:16.035953 15559 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 521b81a530ed46deab1049ac1680b407: Bootstrap starting.
I20260809 02:20:16.038224 15559 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 521b81a530ed46deab1049ac1680b407: Neither blocks nor log segments found. Creating new log.
I20260809 02:20:16.039050 15559 log.cc:824] T 00000000000000000000000000000000 P 521b81a530ed46deab1049ac1680b407: Log is configured to *not* fsync() on all Append() calls
I20260809 02:20:16.039237 15563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:16.037620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.76:43027 (local address 127.0.0.1:45797)
0809 02:20:16.037725 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:16.037731 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:16.037917 (+ 186us) server_negotiation.cc:407] Connection header received
0809 02:20:16.038050 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:16.038053 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:16.038087 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:16.038170 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:16.038861 (+ 691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:20:16.038865 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:20:16.038868 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:20:16.038879 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:20:16.038909 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:20:16.038919 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:20:16.039008 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:16.039090 (+ 82us) server_negotiation.cc:299] Negotiation successful
0809 02:20:16.039129 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 02:20:16.041003 15559 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 521b81a530ed46deab1049ac1680b407: No bootstrap required, opened a new log
I20260809 02:20:16.044256 15559 raft_consensus.cc:348] T 00000000000000000000000000000000 P 521b81a530ed46deab1049ac1680b407 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "521b81a530ed46deab1049ac1680b407" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45797 } }
I20260809 02:20:16.044459 15559 raft_consensus.cc:374] T 00000000000000000000000000000000 P 521b81a530ed46deab1049ac1680b407 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:20:16.044535 15559 raft_consensus.cc:729] T 00000000000000000000000000000000 P 521b81a530ed46deab1049ac1680b407 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 521b81a530ed46deab1049ac1680b407, State: Initialized, Role: FOLLOWER
I20260809 02:20:16.045065 15559 consensus_queue.cc:260] T 00000000000000000000000000000000 P 521b81a530ed46deab1049ac1680b407 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "521b81a530ed46deab1049ac1680b407" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45797 } }
I20260809 02:20:16.045224 15559 raft_consensus.cc:388] T 00000000000000000000000000000000 P 521b81a530ed46deab1049ac1680b407 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:20:16.045293 15559 raft_consensus.cc:482] T 00000000000000000000000000000000 P 521b81a530ed46deab1049ac1680b407 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:20:16.045387 15559 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 521b81a530ed46deab1049ac1680b407 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:20:16.046217 15559 raft_consensus.cc:504] T 00000000000000000000000000000000 P 521b81a530ed46deab1049ac1680b407 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "521b81a530ed46deab1049ac1680b407" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45797 } }
I20260809 02:20:16.046548 15559 leader_election.cc:302] T 00000000000000000000000000000000 P 521b81a530ed46deab1049ac1680b407 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 521b81a530ed46deab1049ac1680b407; no voters:
I20260809 02:20:16.046838 15559 leader_election.cc:288] T 00000000000000000000000000000000 P 521b81a530ed46deab1049ac1680b407 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:20:16.046939 15571 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 521b81a530ed46deab1049ac1680b407 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:20:16.047181 15571 raft_consensus.cc:686] T 00000000000000000000000000000000 P 521b81a530ed46deab1049ac1680b407 [term 1 LEADER]: Becoming Leader. State: Replica: 521b81a530ed46deab1049ac1680b407, State: Running, Role: LEADER
I20260809 02:20:16.047490 15571 consensus_queue.cc:237] T 00000000000000000000000000000000 P 521b81a530ed46deab1049ac1680b407 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "521b81a530ed46deab1049ac1680b407" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45797 } }
I20260809 02:20:16.048050 15559 sys_catalog.cc:564] T 00000000000000000000000000000000 P 521b81a530ed46deab1049ac1680b407 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:20:16.049170 15572 sys_catalog.cc:455] T 00000000000000000000000000000000 P 521b81a530ed46deab1049ac1680b407 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "521b81a530ed46deab1049ac1680b407" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "521b81a530ed46deab1049ac1680b407" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45797 } } }
I20260809 02:20:16.049322 15572 sys_catalog.cc:458] T 00000000000000000000000000000000 P 521b81a530ed46deab1049ac1680b407 [sys.catalog]: This master's current role is: LEADER
I20260809 02:20:16.049677 15574 sys_catalog.cc:455] T 00000000000000000000000000000000 P 521b81a530ed46deab1049ac1680b407 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 521b81a530ed46deab1049ac1680b407. Latest consensus state: current_term: 1 leader_uuid: "521b81a530ed46deab1049ac1680b407" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "521b81a530ed46deab1049ac1680b407" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45797 } } }
I20260809 02:20:16.049769 15574 sys_catalog.cc:458] T 00000000000000000000000000000000 P 521b81a530ed46deab1049ac1680b407 [sys.catalog]: This master's current role is: LEADER
I20260809 02:20:16.049855 15583 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:20:16.052600 15583 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:20:16.058777 15583 catalog_manager.cc:1347] Generated new cluster ID: 8501c824974f48a6968b6d153d8a3d4f
I20260809 02:20:16.058869 15583 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:20:16.074519 15583 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:20:16.074855 15583 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:20:16.084997 15583 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 521b81a530ed46deab1049ac1680b407: Generated new TSK 0
I20260809 02:20:16.085717 15583 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:20:16.100090 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 15458
2026-08-09T02:20:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1593 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241741788589-15662-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:20:16 dist-test-slave-s11t krb5kdc[15597](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:20:16 dist-test-slave-s11t krb5kdc[15597](info): set up 2 sockets
Aug 09 02:20:16 dist-test-slave-s11t krb5kdc[15597](info): commencing operation
krb5kdc: starting...
W20260809 02:20:17.344780 15662 mini_kdc.cc:121] Time spent starting KDC: real 1.219s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Aug 09 02:20:17 dist-test-slave-s11t krb5kdc[15597](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786242017, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:20:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:20:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241741788589-15662-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241741788589-15662-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 02:20:17.464970 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241741788589-15662-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241741788589-15662-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241741788589-15662-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39855
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241741788589-15662-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39855
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:20:17.575734 15613 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:20:17.576006 15613 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:20:17.576095 15613 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:20:17.580154 15613 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:20:17.580232 15613 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:20:17.580271 15613 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:20:17.580327 15613 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:20:17.580372 15613 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:20:17.585531 15613 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241741788589-15662-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241741788589-15662-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39855
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241741788589-15662-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39855
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15613
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241741788589-15662-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:20:17.586484 15613 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:20:17.587455 15613 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:20:17.595847 15619 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:20:17.595856 15620 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:20:17.596032 15613 server_base.cc:1034] running on GCE node
W20260809 02:20:17.595888 15629 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:20:17.596755 15613 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:20:17.597769 15613 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:20:17.598956 15613 hybrid_clock.cc:648] HybridClock initialized: now 1786242017598939 us; error 29 us; skew 500 ppm
Aug 09 02:20:17 dist-test-slave-s11t krb5kdc[15597](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786242017, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:20:17.601694 15613 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 02:20:17.602574 15613 webserver.cc:460] Webserver started at http://127.0.0.1:43315/ using document root <none> and password file <none>
I20260809 02:20:17.603119 15613 fs_manager.cc:359] Metadata directory not provided
I20260809 02:20:17.603211 15613 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:20:17.603451 15613 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:20:17.605494 15613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241741788589-15662-0/minicluster-data/master-0/data/instance:
uuid: "08be8619e3294ad3993d7cdcafd6c220"
format_stamp: "Formatted at 2026-08-09 02:20:17 on dist-test-slave-s11t"
server_key: "5536042065e7b469fa7a62f3c581f629"
server_key_iv: "0770b0aa17afbd0c223234674c8e3492"
server_key_version: "encryptionkey@0"
I20260809 02:20:17.606094 15613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241741788589-15662-0/minicluster-data/master-0/wal/instance:
uuid: "08be8619e3294ad3993d7cdcafd6c220"
format_stamp: "Formatted at 2026-08-09 02:20:17 on dist-test-slave-s11t"
server_key: "5536042065e7b469fa7a62f3c581f629"
server_key_iv: "0770b0aa17afbd0c223234674c8e3492"
server_key_version: "encryptionkey@0"
I20260809 02:20:17.609692 15613 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 02:20:17.612038 15636 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:20:17.613292 15613 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 02:20:17.613440 15613 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241741788589-15662-0/minicluster-data/master-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241741788589-15662-0/minicluster-data/master-0/wal
uuid: "08be8619e3294ad3993d7cdcafd6c220"
format_stamp: "Formatted at 2026-08-09 02:20:17 on dist-test-slave-s11t"
server_key: "5536042065e7b469fa7a62f3c581f629"
server_key_iv: "0770b0aa17afbd0c223234674c8e3492"
server_key_version: "encryptionkey@0"
I20260809 02:20:17.613549 15613 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241741788589-15662-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241741788589-15662-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241741788589-15662-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:20:17.622869 15613 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:20:17.625524 15613 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:20:17.625720 15613 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:20:17.632761 15613 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39855
I20260809 02:20:17.632788 15730 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39855 every 8 connection(s)
I20260809 02:20:17.633710 15613 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
I20260809 02:20:17.636307 15734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:20:17.640821 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 15613
I20260809 02:20:17.640978 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241741788589-15662-0/minicluster-data/master-0/wal/instance
I20260809 02:20:17.641280 15662 external_mini_cluster.cc:1428] Setting key 7f1c2e0a4fcd9e43d05048d9efabdc03
I20260809 02:20:17.641945 15734 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08be8619e3294ad3993d7cdcafd6c220: Bootstrap starting.
I20260809 02:20:17.644497 15734 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 08be8619e3294ad3993d7cdcafd6c220: Neither blocks nor log segments found. Creating new log.
I20260809 02:20:17.645373 15734 log.cc:824] T 00000000000000000000000000000000 P 08be8619e3294ad3993d7cdcafd6c220: Log is configured to *not* fsync() on all Append() calls
I20260809 02:20:17.647397 15734 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08be8619e3294ad3993d7cdcafd6c220: No bootstrap required, opened a new log
Aug 09 02:20:17 dist-test-slave-s11t krb5kdc[15597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786242017, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 02:20:17.651142 15734 raft_consensus.cc:348] T 00000000000000000000000000000000 P 08be8619e3294ad3993d7cdcafd6c220 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08be8619e3294ad3993d7cdcafd6c220" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39855 } }
I20260809 02:20:17.651417 15734 raft_consensus.cc:374] T 00000000000000000000000000000000 P 08be8619e3294ad3993d7cdcafd6c220 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:20:17.651535 15734 raft_consensus.cc:729] T 00000000000000000000000000000000 P 08be8619e3294ad3993d7cdcafd6c220 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08be8619e3294ad3993d7cdcafd6c220, State: Initialized, Role: FOLLOWER
I20260809 02:20:17.652077 15734 consensus_queue.cc:260] T 00000000000000000000000000000000 P 08be8619e3294ad3993d7cdcafd6c220 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08be8619e3294ad3993d7cdcafd6c220" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39855 } }
I20260809 02:20:17.652263 15734 raft_consensus.cc:388] T 00000000000000000000000000000000 P 08be8619e3294ad3993d7cdcafd6c220 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:20:17.652344 15734 raft_consensus.cc:482] T 00000000000000000000000000000000 P 08be8619e3294ad3993d7cdcafd6c220 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:20:17.652459 15734 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 08be8619e3294ad3993d7cdcafd6c220 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:20:17.653546 15734 raft_consensus.cc:504] T 00000000000000000000000000000000 P 08be8619e3294ad3993d7cdcafd6c220 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08be8619e3294ad3993d7cdcafd6c220" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39855 } }
I20260809 02:20:17.653869 15734 leader_election.cc:302] T 00000000000000000000000000000000 P 08be8619e3294ad3993d7cdcafd6c220 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 08be8619e3294ad3993d7cdcafd6c220; no voters:
I20260809 02:20:17.654168 15734 leader_election.cc:288] T 00000000000000000000000000000000 P 08be8619e3294ad3993d7cdcafd6c220 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:20:17.654277 15742 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 08be8619e3294ad3993d7cdcafd6c220 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:20:17.654533 15742 raft_consensus.cc:686] T 00000000000000000000000000000000 P 08be8619e3294ad3993d7cdcafd6c220 [term 1 LEADER]: Becoming Leader. State: Replica: 08be8619e3294ad3993d7cdcafd6c220, State: Running, Role: LEADER
I20260809 02:20:17.654893 15742 consensus_queue.cc:237] T 00000000000000000000000000000000 P 08be8619e3294ad3993d7cdcafd6c220 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08be8619e3294ad3993d7cdcafd6c220" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39855 } }
I20260809 02:20:17.655341 15734 sys_catalog.cc:564] T 00000000000000000000000000000000 P 08be8619e3294ad3993d7cdcafd6c220 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:20:17.656059 15745 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08be8619e3294ad3993d7cdcafd6c220 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 08be8619e3294ad3993d7cdcafd6c220. Latest consensus state: current_term: 1 leader_uuid: "08be8619e3294ad3993d7cdcafd6c220" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08be8619e3294ad3993d7cdcafd6c220" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39855 } } }
I20260809 02:20:17.656255 15745 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08be8619e3294ad3993d7cdcafd6c220 [sys.catalog]: This master's current role is: LEADER
I20260809 02:20:17.656652 15755 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:20:17.657024 15740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:17.642716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.76:45673 (local address 127.0.0.1:39855)
0809 02:20:17.643147 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:17.643165 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:17.643207 (+ 42us) server_negotiation.cc:407] Connection header received
0809 02:20:17.644007 (+ 800us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:17.644030 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:17.644459 (+ 429us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:17.644846 (+ 387us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:17.645515 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:17.646878 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:17.647774 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:17.648435 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:17.648588 (+ 153us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:17.651135 (+ 2547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:20:17.651141 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:20:17.651156 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:20:17.651191 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:20:17.653262 (+ 2071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:17.653768 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:17.653774 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:17.653779 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:17.653905 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:17.654513 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:17.654518 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:17.654519 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:17.655069 (+ 550us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:20:17.655277 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:17.655764 (+ 487us) server_negotiation.cc:299] Negotiation successful
0809 02:20:17.655979 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":102,"threads_started":1}
I20260809 02:20:17.658330 15744 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08be8619e3294ad3993d7cdcafd6c220 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "08be8619e3294ad3993d7cdcafd6c220" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08be8619e3294ad3993d7cdcafd6c220" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39855 } } }
I20260809 02:20:17.658519 15744 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08be8619e3294ad3993d7cdcafd6c220 [sys.catalog]: This master's current role is: LEADER
I20260809 02:20:17.660086 15755 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:20:17.660414 15662 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:20:17.665795 15755 catalog_manager.cc:1347] Generated new cluster ID: 43d66229db4442afa2dce6c44c751dd1
I20260809 02:20:17.665858 15755 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:20:17.671993 15740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:17.662646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.76:48653 (local address 127.0.0.1:39855)
0809 02:20:17.662765 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:17.662769 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:17.663085 (+ 316us) server_negotiation.cc:407] Connection header received
0809 02:20:17.663204 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:17.663206 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:17.663253 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:17.663339 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:17.664149 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:17.664800 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:17.665666 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:17.666686 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:17.667146 (+ 460us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:17.668363 (+ 1217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:20:17.668367 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:20:17.668369 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:20:17.668393 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:20:17.669989 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:17.670512 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:17.670515 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:17.670517 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:17.670585 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:17.670915 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:17.670917 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:17.670919 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:17.671089 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:20:17.671672 (+ 583us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:17.671763 (+ 91us) server_negotiation.cc:299] Negotiation successful
0809 02:20:17.671843 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 02:20:17.675137 15755 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:20:17.675932 15755 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:20:17.683430 15755 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 08be8619e3294ad3993d7cdcafd6c220: Generated new TSK 0
I20260809 02:20:17.684054 15755 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:20:17.688647 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 15613
W20260809 02:20:17.701640 15595 connection.cc:535] client connection to 127.0.0.1:39855 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39855): Connection reset by peer (error 104)
2026-08-09T02:20:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1590 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241741788589-15662-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:20:17 dist-test-slave-s11t krb5kdc[15781](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:20:17 dist-test-slave-s11t krb5kdc[15781](info): set up 2 sockets
Aug 09 02:20:17 dist-test-slave-s11t krb5kdc[15781](info): commencing operation
krb5kdc: starting...
W20260809 02:20:18.921929 15662 mini_kdc.cc:121] Time spent starting KDC: real 1.206s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Aug 09 02:20:19 dist-test-slave-s11t krb5kdc[15781](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786242019, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:20:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:20:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241741788589-15662-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241741788589-15662-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 02:20:19.047619 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241741788589-15662-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241741788589-15662-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241741788589-15662-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42429
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241741788589-15662-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42429
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:20:19.158713 15803 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:20:19.158977 15803 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:20:19.159017 15803 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:20:19.163069 15803 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:20:19.163125 15803 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:20:19.163143 15803 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:20:19.163160 15803 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:20:19.163215 15803 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:20:19.168292 15803 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241741788589-15662-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241741788589-15662-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42429
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241741788589-15662-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42429
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15803
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241741788589-15662-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:20:19.169287 15803 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:20:19.170279 15803 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:20:19.178807 15808 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:20:19.178853 15810 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:20:19.178992 15803 server_base.cc:1034] running on GCE node
W20260809 02:20:19.179147 15813 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:20:19.179716 15803 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:20:19.180672 15803 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:20:19.181900 15803 hybrid_clock.cc:648] HybridClock initialized: now 1786242019181892 us; error 32 us; skew 500 ppm
Aug 09 02:20:19 dist-test-slave-s11t krb5kdc[15781](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786242019, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:20:19.184573 15803 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 02:20:19.185436 15803 webserver.cc:460] Webserver started at http://127.0.0.1:35789/ using document root <none> and password file <none>
I20260809 02:20:19.185937 15803 fs_manager.cc:359] Metadata directory not provided
I20260809 02:20:19.185997 15803 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:20:19.186154 15803 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:20:19.188051 15803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241741788589-15662-0/minicluster-data/master-0/data/instance:
uuid: "bdf631c581084bb2940c38bf9c093f41"
format_stamp: "Formatted at 2026-08-09 02:20:19 on dist-test-slave-s11t"
server_key: "b6b90dc922a472a3da300d5d597d0ff7"
server_key_iv: "512abbb08ee9f29191acd31df8b4fa6c"
server_key_version: "encryptionkey@0"
I20260809 02:20:19.188552 15803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241741788589-15662-0/minicluster-data/master-0/wal/instance:
uuid: "bdf631c581084bb2940c38bf9c093f41"
format_stamp: "Formatted at 2026-08-09 02:20:19 on dist-test-slave-s11t"
server_key: "b6b90dc922a472a3da300d5d597d0ff7"
server_key_iv: "512abbb08ee9f29191acd31df8b4fa6c"
server_key_version: "encryptionkey@0"
I20260809 02:20:19.192062 15803 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.001s
I20260809 02:20:19.194437 15822 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:20:19.195555 15803 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260809 02:20:19.195698 15803 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241741788589-15662-0/minicluster-data/master-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241741788589-15662-0/minicluster-data/master-0/wal
uuid: "bdf631c581084bb2940c38bf9c093f41"
format_stamp: "Formatted at 2026-08-09 02:20:19 on dist-test-slave-s11t"
server_key: "b6b90dc922a472a3da300d5d597d0ff7"
server_key_iv: "512abbb08ee9f29191acd31df8b4fa6c"
server_key_version: "encryptionkey@0"
I20260809 02:20:19.195797 15803 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241741788589-15662-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241741788589-15662-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241741788589-15662-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:20:19.210469 15803 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:20:19.213187 15803 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:20:19.213371 15803 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:20:19.220249 15803 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42429
I20260809 02:20:19.220276 15893 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42429 every 8 connection(s)
I20260809 02:20:19.221194 15803 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
I20260809 02:20:19.223453 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 15803
I20260809 02:20:19.223598 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241741788589-15662-0/minicluster-data/master-0/wal/instance
I20260809 02:20:19.223887 15662 external_mini_cluster.cc:1428] Setting key 9c9327e3088e5889f01a2777735725dd
I20260809 02:20:19.224050 15895 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:20:19.231258 15895 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bdf631c581084bb2940c38bf9c093f41: Bootstrap starting.
I20260809 02:20:19.234030 15895 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bdf631c581084bb2940c38bf9c093f41: Neither blocks nor log segments found. Creating new log.
I20260809 02:20:19.234131 15898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:19.225295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.76:56865 (local address 127.0.0.1:42429)
0809 02:20:19.225710 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:19.225729 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:19.225770 (+ 41us) server_negotiation.cc:407] Connection header received
0809 02:20:19.226815 (+ 1045us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:19.226840 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:19.227277 (+ 437us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:19.227634 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:19.228035 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:19.229599 (+ 1564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:19.230465 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:19.231129 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:19.231311 (+ 182us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:19.232195 (+ 884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:20:19.232202 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:20:19.232216 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:20:19.232741 (+ 525us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:20:19.232812 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:20:19.232842 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:20:19.232974 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:19.233124 (+ 150us) server_negotiation.cc:299] Negotiation successful
0809 02:20:19.233256 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":112,"threads_started":1}
I20260809 02:20:19.234933 15895 log.cc:824] T 00000000000000000000000000000000 P bdf631c581084bb2940c38bf9c093f41: Log is configured to *not* fsync() on all Append() calls
I20260809 02:20:19.237131 15895 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bdf631c581084bb2940c38bf9c093f41: No bootstrap required, opened a new log
I20260809 02:20:19.237185 15662 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:20:19.240795 15895 raft_consensus.cc:348] T 00000000000000000000000000000000 P bdf631c581084bb2940c38bf9c093f41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdf631c581084bb2940c38bf9c093f41" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42429 } }
I20260809 02:20:19.241008 15895 raft_consensus.cc:374] T 00000000000000000000000000000000 P bdf631c581084bb2940c38bf9c093f41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:20:19.241084 15895 raft_consensus.cc:729] T 00000000000000000000000000000000 P bdf631c581084bb2940c38bf9c093f41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdf631c581084bb2940c38bf9c093f41, State: Initialized, Role: FOLLOWER
I20260809 02:20:19.241498 15895 consensus_queue.cc:260] T 00000000000000000000000000000000 P bdf631c581084bb2940c38bf9c093f41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdf631c581084bb2940c38bf9c093f41" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42429 } }
I20260809 02:20:19.241650 15895 raft_consensus.cc:388] T 00000000000000000000000000000000 P bdf631c581084bb2940c38bf9c093f41 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:20:19.241742 15895 raft_consensus.cc:482] T 00000000000000000000000000000000 P bdf631c581084bb2940c38bf9c093f41 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:20:19.241859 15895 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bdf631c581084bb2940c38bf9c093f41 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:20:19.242873 15895 raft_consensus.cc:504] T 00000000000000000000000000000000 P bdf631c581084bb2940c38bf9c093f41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdf631c581084bb2940c38bf9c093f41" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42429 } }
I20260809 02:20:19.243256 15895 leader_election.cc:302] T 00000000000000000000000000000000 P bdf631c581084bb2940c38bf9c093f41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bdf631c581084bb2940c38bf9c093f41; no voters:
I20260809 02:20:19.243439 15898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:19.239171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.76:51267 (local address 127.0.0.1:42429)
0809 02:20:19.239298 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:19.239304 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:19.239502 (+ 198us) server_negotiation.cc:407] Connection header received
0809 02:20:19.239646 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:19.239649 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:19.239697 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:19.239779 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:19.240054 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:19.240541 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:19.241588 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:19.241974 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:19.242078 (+ 104us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:19.242899 (+ 821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:20:19.242903 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:20:19.242906 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:20:19.242918 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:20:19.242946 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:20:19.242962 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:20:19.243057 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:19.243279 (+ 222us) server_negotiation.cc:299] Negotiation successful
0809 02:20:19.243314 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 02:20:19.243587 15895 leader_election.cc:288] T 00000000000000000000000000000000 P bdf631c581084bb2940c38bf9c093f41 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:20:19.243683 15907 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bdf631c581084bb2940c38bf9c093f41 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:20:19.243927 15907 raft_consensus.cc:686] T 00000000000000000000000000000000 P bdf631c581084bb2940c38bf9c093f41 [term 1 LEADER]: Becoming Leader. State: Replica: bdf631c581084bb2940c38bf9c093f41, State: Running, Role: LEADER
I20260809 02:20:19.244239 15907 consensus_queue.cc:237] T 00000000000000000000000000000000 P bdf631c581084bb2940c38bf9c093f41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdf631c581084bb2940c38bf9c093f41" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42429 } }
I20260809 02:20:19.244987 15895 sys_catalog.cc:564] T 00000000000000000000000000000000 P bdf631c581084bb2940c38bf9c093f41 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:20:19.245951 15909 sys_catalog.cc:455] T 00000000000000000000000000000000 P bdf631c581084bb2940c38bf9c093f41 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bdf631c581084bb2940c38bf9c093f41. Latest consensus state: current_term: 1 leader_uuid: "bdf631c581084bb2940c38bf9c093f41" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdf631c581084bb2940c38bf9c093f41" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42429 } } }
I20260809 02:20:19.246089 15909 sys_catalog.cc:458] T 00000000000000000000000000000000 P bdf631c581084bb2940c38bf9c093f41 [sys.catalog]: This master's current role is: LEADER
I20260809 02:20:19.246397 15908 sys_catalog.cc:455] T 00000000000000000000000000000000 P bdf631c581084bb2940c38bf9c093f41 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bdf631c581084bb2940c38bf9c093f41" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdf631c581084bb2940c38bf9c093f41" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42429 } } }
I20260809 02:20:19.246490 15908 sys_catalog.cc:458] T 00000000000000000000000000000000 P bdf631c581084bb2940c38bf9c093f41 [sys.catalog]: This master's current role is: LEADER
I20260809 02:20:19.247237 15924 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:20:19.250555 15924 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:20:19.256040 15924 catalog_manager.cc:1347] Generated new cluster ID: 080ace3bd33c426ea7027bff441e6589
I20260809 02:20:19.256114 15924 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:20:19.262817 15924 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:20:19.263633 15924 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:20:19.268045 15924 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bdf631c581084bb2940c38bf9c093f41: Generated new TSK 0
I20260809 02:20:19.268646 15924 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:20:19.273919 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 15803
W20260809 02:20:19.293601 15777 connection.cc:535] client connection to 127.0.0.1:42429 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42429): Connection reset by peer (error 104)
2026-08-09T02:20:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1591 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241741788589-15662-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:20:19 dist-test-slave-s11t krb5kdc[15942](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:20:19 dist-test-slave-s11t krb5kdc[15942](info): set up 2 sockets
Aug 09 02:20:19 dist-test-slave-s11t krb5kdc[15942](info): commencing operation
krb5kdc: starting...
W20260809 02:20:20.525147 15662 mini_kdc.cc:121] Time spent starting KDC: real 1.217s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Aug 09 02:20:20 dist-test-slave-s11t krb5kdc[15942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786242020, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:20:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:20:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241741788589-15662-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241741788589-15662-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 02:20:20.645987 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241741788589-15662-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241741788589-15662-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241741788589-15662-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36353
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241741788589-15662-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36353
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:20:20.755355 15962 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:20:20.755620 15962 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:20:20.755690 15962 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:20:20.759649 15962 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:20:20.759737 15962 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:20:20.759773 15962 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:20:20.759833 15962 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:20:20.759867 15962 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:20:20.764981 15962 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241741788589-15662-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241741788589-15662-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36353
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241741788589-15662-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36353
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15962
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241741788589-15662-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:20:20.765957 15962 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:20:20.766947 15962 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:20:20.775395 15970 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:20:20.775377 15972 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:20:20.775358 15962 server_base.cc:1034] running on GCE node
W20260809 02:20:20.775339 15969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:20:20.776263 15962 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:20:20.777321 15962 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:20:20.778541 15962 hybrid_clock.cc:648] HybridClock initialized: now 1786242020778510 us; error 43 us; skew 500 ppm
Aug 09 02:20:20 dist-test-slave-s11t krb5kdc[15942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786242020, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:20:20.786013 15962 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 02:20:20.786865 15962 webserver.cc:460] Webserver started at http://127.0.0.1:36105/ using document root <none> and password file <none>
I20260809 02:20:20.787415 15962 fs_manager.cc:359] Metadata directory not provided
I20260809 02:20:20.787506 15962 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:20:20.787738 15962 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:20:20.789767 15962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241741788589-15662-0/minicluster-data/master-0/data/instance:
uuid: "545cf941223f433b83edcefe53e6ce43"
format_stamp: "Formatted at 2026-08-09 02:20:20 on dist-test-slave-s11t"
server_key: "e1eaba9e3c12c0c1116f3c81da874c1f"
server_key_iv: "2409a4d03cc093be0d8471d44fe8fdbd"
server_key_version: "encryptionkey@0"
I20260809 02:20:20.790340 15962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241741788589-15662-0/minicluster-data/master-0/wal/instance:
uuid: "545cf941223f433b83edcefe53e6ce43"
format_stamp: "Formatted at 2026-08-09 02:20:20 on dist-test-slave-s11t"
server_key: "e1eaba9e3c12c0c1116f3c81da874c1f"
server_key_iv: "2409a4d03cc093be0d8471d44fe8fdbd"
server_key_version: "encryptionkey@0"
I20260809 02:20:20.793865 15962 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 02:20:20.796064 15981 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:20:20.797237 15962 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 02:20:20.797374 15962 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241741788589-15662-0/minicluster-data/master-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241741788589-15662-0/minicluster-data/master-0/wal
uuid: "545cf941223f433b83edcefe53e6ce43"
format_stamp: "Formatted at 2026-08-09 02:20:20 on dist-test-slave-s11t"
server_key: "e1eaba9e3c12c0c1116f3c81da874c1f"
server_key_iv: "2409a4d03cc093be0d8471d44fe8fdbd"
server_key_version: "encryptionkey@0"
I20260809 02:20:20.797482 15962 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241741788589-15662-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241741788589-15662-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241741788589-15662-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:20:20.801849 15962 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:20:20.805007 15962 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:20:20.805197 15962 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:20:20.812032 15962 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36353
I20260809 02:20:20.812091 16062 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36353 every 8 connection(s)
I20260809 02:20:20.813009 15962 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
I20260809 02:20:20.815604 16063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:20:20.820891 16063 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 545cf941223f433b83edcefe53e6ce43: Bootstrap starting.
I20260809 02:20:20.821689 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 15962
I20260809 02:20:20.821867 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241741788589-15662-0/minicluster-data/master-0/wal/instance
I20260809 02:20:20.822147 15662 external_mini_cluster.cc:1428] Setting key cbc090b41638eaeb3b4516abf0ad6635
I20260809 02:20:20.823639 16063 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 545cf941223f433b83edcefe53e6ce43: Neither blocks nor log segments found. Creating new log.
I20260809 02:20:20.824350 16063 log.cc:824] T 00000000000000000000000000000000 P 545cf941223f433b83edcefe53e6ce43: Log is configured to *not* fsync() on all Append() calls
I20260809 02:20:20.826567 16063 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 545cf941223f433b83edcefe53e6ce43: No bootstrap required, opened a new log
I20260809 02:20:20.827162 16067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:20.823240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.76:57507 (local address 127.0.0.1:36353)
0809 02:20:20.823656 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:20.823671 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:20.823706 (+ 35us) server_negotiation.cc:407] Connection header received
0809 02:20:20.824379 (+ 673us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:20.824395 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:20.824772 (+ 377us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:20.825150 (+ 378us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:20.825940 (+ 790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:20:20.825945 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:20:20.825956 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:20:20.825966 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:20:20.826015 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:20:20.826033 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:20:20.826120 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:20.826267 (+ 147us) server_negotiation.cc:299] Negotiation successful
0809 02:20:20.826438 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":127,"threads_started":1}
I20260809 02:20:20.829865 15662 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:20:20.830586 16063 raft_consensus.cc:348] T 00000000000000000000000000000000 P 545cf941223f433b83edcefe53e6ce43 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "545cf941223f433b83edcefe53e6ce43" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36353 } }
I20260809 02:20:20.830806 16063 raft_consensus.cc:374] T 00000000000000000000000000000000 P 545cf941223f433b83edcefe53e6ce43 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:20:20.830892 16063 raft_consensus.cc:729] T 00000000000000000000000000000000 P 545cf941223f433b83edcefe53e6ce43 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 545cf941223f433b83edcefe53e6ce43, State: Initialized, Role: FOLLOWER
I20260809 02:20:20.831370 16063 consensus_queue.cc:260] T 00000000000000000000000000000000 P 545cf941223f433b83edcefe53e6ce43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "545cf941223f433b83edcefe53e6ce43" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36353 } }
I20260809 02:20:20.831523 16063 raft_consensus.cc:388] T 00000000000000000000000000000000 P 545cf941223f433b83edcefe53e6ce43 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:20:20.831593 16063 raft_consensus.cc:482] T 00000000000000000000000000000000 P 545cf941223f433b83edcefe53e6ce43 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:20:20.831691 16063 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 545cf941223f433b83edcefe53e6ce43 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:20:20.832839 16063 raft_consensus.cc:504] T 00000000000000000000000000000000 P 545cf941223f433b83edcefe53e6ce43 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "545cf941223f433b83edcefe53e6ce43" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36353 } }
I20260809 02:20:20.833226 16063 leader_election.cc:302] T 00000000000000000000000000000000 P 545cf941223f433b83edcefe53e6ce43 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 545cf941223f433b83edcefe53e6ce43; no voters:
I20260809 02:20:20.833590 16063 leader_election.cc:288] T 00000000000000000000000000000000 P 545cf941223f433b83edcefe53e6ce43 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:20:20.833683 16076 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 545cf941223f433b83edcefe53e6ce43 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:20:20.833808 16067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:20.831925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.76:46931 (local address 127.0.0.1:36353)
0809 02:20:20.832058 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:20.832062 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:20.832263 (+ 201us) server_negotiation.cc:407] Connection header received
0809 02:20:20.832370 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:20.832372 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:20.832399 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:20.832462 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:20.833191 (+ 729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:20:20.833194 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:20:20.833196 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:20:20.833206 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:20:20.833229 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:20:20.833238 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:20:20.833309 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:20.833591 (+ 282us) server_negotiation.cc:299] Negotiation successful
0809 02:20:20.833632 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 02:20:20.833985 16076 raft_consensus.cc:686] T 00000000000000000000000000000000 P 545cf941223f433b83edcefe53e6ce43 [term 1 LEADER]: Becoming Leader. State: Replica: 545cf941223f433b83edcefe53e6ce43, State: Running, Role: LEADER
I20260809 02:20:20.834317 16076 consensus_queue.cc:237] T 00000000000000000000000000000000 P 545cf941223f433b83edcefe53e6ce43 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "545cf941223f433b83edcefe53e6ce43" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36353 } }
I20260809 02:20:20.834854 16063 sys_catalog.cc:564] T 00000000000000000000000000000000 P 545cf941223f433b83edcefe53e6ce43 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:20:20.835996 16078 sys_catalog.cc:455] T 00000000000000000000000000000000 P 545cf941223f433b83edcefe53e6ce43 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 545cf941223f433b83edcefe53e6ce43. Latest consensus state: current_term: 1 leader_uuid: "545cf941223f433b83edcefe53e6ce43" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "545cf941223f433b83edcefe53e6ce43" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36353 } } }
I20260809 02:20:20.836125 16078 sys_catalog.cc:458] T 00000000000000000000000000000000 P 545cf941223f433b83edcefe53e6ce43 [sys.catalog]: This master's current role is: LEADER
I20260809 02:20:20.836426 16077 sys_catalog.cc:455] T 00000000000000000000000000000000 P 545cf941223f433b83edcefe53e6ce43 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "545cf941223f433b83edcefe53e6ce43" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "545cf941223f433b83edcefe53e6ce43" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36353 } } }
I20260809 02:20:20.836530 16077 sys_catalog.cc:458] T 00000000000000000000000000000000 P 545cf941223f433b83edcefe53e6ce43 [sys.catalog]: This master's current role is: LEADER
W20260809 02:20:20.839411 16093 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 545cf941223f433b83edcefe53e6ce43: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:20:20.839499 16093 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:20:20.839586 16086 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:20:20.840385 16086 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:20:20.845863 16086 catalog_manager.cc:1347] Generated new cluster ID: 55c82adcff9b40a684af4e83e8958ece
I20260809 02:20:20.845942 16086 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:20:20.853322 16086 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:20:20.853626 16086 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:20:20.869040 16086 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 545cf941223f433b83edcefe53e6ce43: Generated new TSK 0
I20260809 02:20:20.869822 16086 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:20:20.880191 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 15962
2026-08-09T02:20:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1599 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241741788589-15662-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:20:20 dist-test-slave-s11t krb5kdc[16103](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:20:20 dist-test-slave-s11t krb5kdc[16103](info): set up 2 sockets
Aug 09 02:20:20 dist-test-slave-s11t krb5kdc[16103](info): commencing operation
krb5kdc: starting...
W20260809 02:20:22.155789 15662 mini_kdc.cc:121] Time spent starting KDC: real 1.248s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Aug 09 02:20:22 dist-test-slave-s11t krb5kdc[16103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786242022, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:20:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:20:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.75.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab.
Entry for principal kudu/127.15.75.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab.
I20260809 02:20:22.285512 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241741788589-15662-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241741788589-15662-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241741788589-15662-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:37653
--webserver_interface=127.15.75.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab
--principal=kudu/127.15.75.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:43321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.75.190:37653
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:20:22.396545 16124 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:20:22.396840 16124 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:20:22.396883 16124 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:20:22.400847 16124 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:20:22.400900 16124 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:20:22.400920 16124 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:20:22.400936 16124 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:20:22.400952 16124 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:20:22.406023 16124 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:43321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241741788589-15662-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241741788589-15662-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.75.190:37653
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab
--principal=kudu/127.15.75.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:37653
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.75.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16124
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241741788589-15662-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:20:22.406988 16124 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:20:22.407960 16124 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:20:22.416245 16139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:20:22.416267 16136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:20:22.416438 16124 server_base.cc:1034] running on GCE node
W20260809 02:20:22.416267 16134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:20:22.417088 16124 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:20:22.418062 16124 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:20:22.419230 16124 hybrid_clock.cc:648] HybridClock initialized: now 1786242022419212 us; error 30 us; skew 500 ppm
Aug 09 02:20:22 dist-test-slave-s11t krb5kdc[16103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786242022, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:20:22.421958 16124 init.cc:377] Logged in from keytab as kudu/127.15.75.190@KRBTEST.COM (short username kudu)
I20260809 02:20:22.422739 16124 webserver.cc:460] Webserver started at http://127.15.75.190:44169/ using document root <none> and password file <none>
I20260809 02:20:22.423270 16124 fs_manager.cc:359] Metadata directory not provided
I20260809 02:20:22.423332 16124 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:20:22.423496 16124 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:20:22.425479 16124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241741788589-15662-0/minicluster-data/master-0/data/instance:
uuid: "1329b85e557d4f5282015dc59efffd30"
format_stamp: "Formatted at 2026-08-09 02:20:22 on dist-test-slave-s11t"
server_key: "989b8618e006269a74d2b1464d9803b2"
server_key_iv: "4ad1cbb57f9538523129aec9ad6d13e9"
server_key_version: "encryptionkey@0"
I20260809 02:20:22.425992 16124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241741788589-15662-0/minicluster-data/master-0/wal/instance:
uuid: "1329b85e557d4f5282015dc59efffd30"
format_stamp: "Formatted at 2026-08-09 02:20:22 on dist-test-slave-s11t"
server_key: "989b8618e006269a74d2b1464d9803b2"
server_key_iv: "4ad1cbb57f9538523129aec9ad6d13e9"
server_key_version: "encryptionkey@0"
I20260809 02:20:22.429585 16124 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 02:20:22.431927 16150 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:20:22.433166 16124 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 02:20:22.433264 16124 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241741788589-15662-0/minicluster-data/master-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241741788589-15662-0/minicluster-data/master-0/wal
uuid: "1329b85e557d4f5282015dc59efffd30"
format_stamp: "Formatted at 2026-08-09 02:20:22 on dist-test-slave-s11t"
server_key: "989b8618e006269a74d2b1464d9803b2"
server_key_iv: "4ad1cbb57f9538523129aec9ad6d13e9"
server_key_version: "encryptionkey@0"
I20260809 02:20:22.433346 16124 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241741788589-15662-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241741788589-15662-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241741788589-15662-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:20:22.444656 16124 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:20:22.447427 16124 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:20:22.447568 16124 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:20:22.454306 16124 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.190:37653
I20260809 02:20:22.454345 16217 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.190:37653 every 8 connection(s)
I20260809 02:20:22.455192 16124 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
I20260809 02:20:22.457940 16219 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:20:22.461102 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 16124
I20260809 02:20:22.461270 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241741788589-15662-0/minicluster-data/master-0/wal/instance
I20260809 02:20:22.461572 15662 external_mini_cluster.cc:1428] Setting key b2b1ac32ca2c0cb05ef89b6c67b22998
I20260809 02:20:22.463766 16219 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1329b85e557d4f5282015dc59efffd30: Bootstrap starting.
I20260809 02:20:22.466063 16219 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1329b85e557d4f5282015dc59efffd30: Neither blocks nor log segments found. Creating new log.
I20260809 02:20:22.466816 16219 log.cc:824] T 00000000000000000000000000000000 P 1329b85e557d4f5282015dc59efffd30: Log is configured to *not* fsync() on all Append() calls
I20260809 02:20:22.468639 16219 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1329b85e557d4f5282015dc59efffd30: No bootstrap required, opened a new log
Aug 09 02:20:22 dist-test-slave-s11t krb5kdc[16103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786242022, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:20:22.472445 16219 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1329b85e557d4f5282015dc59efffd30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1329b85e557d4f5282015dc59efffd30" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 37653 } }
I20260809 02:20:22.472651 16219 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1329b85e557d4f5282015dc59efffd30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:20:22.472749 16219 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1329b85e557d4f5282015dc59efffd30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1329b85e557d4f5282015dc59efffd30, State: Initialized, Role: FOLLOWER
I20260809 02:20:22.473227 16219 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1329b85e557d4f5282015dc59efffd30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1329b85e557d4f5282015dc59efffd30" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 37653 } }
I20260809 02:20:22.473382 16219 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1329b85e557d4f5282015dc59efffd30 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:20:22.473470 16219 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1329b85e557d4f5282015dc59efffd30 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:20:22.473587 16219 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1329b85e557d4f5282015dc59efffd30 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:20:22.474421 16219 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1329b85e557d4f5282015dc59efffd30 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1329b85e557d4f5282015dc59efffd30" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 37653 } }
I20260809 02:20:22.474766 16219 leader_election.cc:302] T 00000000000000000000000000000000 P 1329b85e557d4f5282015dc59efffd30 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1329b85e557d4f5282015dc59efffd30; no voters:
I20260809 02:20:22.475123 16219 leader_election.cc:288] T 00000000000000000000000000000000 P 1329b85e557d4f5282015dc59efffd30 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:20:22.475248 16225 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1329b85e557d4f5282015dc59efffd30 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:20:22.475462 16225 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1329b85e557d4f5282015dc59efffd30 [term 1 LEADER]: Becoming Leader. State: Replica: 1329b85e557d4f5282015dc59efffd30, State: Running, Role: LEADER
I20260809 02:20:22.475744 16225 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1329b85e557d4f5282015dc59efffd30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1329b85e557d4f5282015dc59efffd30" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 37653 } }
I20260809 02:20:22.476205 16222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:22.463007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55113 (local address 127.15.75.190:37653)
0809 02:20:22.463439 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:22.463454 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:22.463487 (+ 33us) server_negotiation.cc:407] Connection header received
0809 02:20:22.464154 (+ 667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:22.464173 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:22.464503 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:22.464874 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:22.465716 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:22.466782 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:22.467702 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:22.468168 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:22.468374 (+ 206us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:22.471034 (+ 2660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:20:22.471039 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:20:22.471052 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:20:22.471078 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:20:22.472999 (+ 1921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:22.473538 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:22.473546 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:22.473553 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:22.473707 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:22.474055 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:22.474059 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:22.474060 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:22.474455 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:20:22.474697 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:22.475027 (+ 330us) server_negotiation.cc:299] Negotiation successful
0809 02:20:22.475279 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":132,"threads_started":1}
I20260809 02:20:22.476378 16219 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1329b85e557d4f5282015dc59efffd30 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:20:22.476903 16226 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1329b85e557d4f5282015dc59efffd30 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1329b85e557d4f5282015dc59efffd30. Latest consensus state: current_term: 1 leader_uuid: "1329b85e557d4f5282015dc59efffd30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1329b85e557d4f5282015dc59efffd30" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 37653 } } }
I20260809 02:20:22.477100 16226 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1329b85e557d4f5282015dc59efffd30 [sys.catalog]: This master's current role is: LEADER
I20260809 02:20:22.477375 16227 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1329b85e557d4f5282015dc59efffd30 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1329b85e557d4f5282015dc59efffd30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1329b85e557d4f5282015dc59efffd30" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 37653 } } }
I20260809 02:20:22.477475 16227 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1329b85e557d4f5282015dc59efffd30 [sys.catalog]: This master's current role is: LEADER
I20260809 02:20:22.477653 16232 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:20:22.479527 15662 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:20:22.481042 16232 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:20:22.487232 16232 catalog_manager.cc:1347] Generated new cluster ID: aaf29f9ee9504e298f2e81533c861f51
I20260809 02:20:22.487293 16232 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:20:22.490993 16222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:22.481473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41053 (local address 127.15.75.190:37653)
0809 02:20:22.481580 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:22.481584 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:22.481770 (+ 186us) server_negotiation.cc:407] Connection header received
0809 02:20:22.481968 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:22.481971 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:22.482012 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:22.482063 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:22.483027 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:22.483932 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:22.485146 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:22.485475 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:22.485591 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:22.487155 (+ 1564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:20:22.487159 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:20:22.487161 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:20:22.487185 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:20:22.488941 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:22.489638 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:22.489643 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:22.489646 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:22.489743 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:22.490113 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:22.490117 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:22.490118 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:22.490297 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:20:22.490456 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:22.490737 (+ 281us) server_negotiation.cc:299] Negotiation successful
0809 02:20:22.490829 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 02:20:22.508391 16232 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:20:22.509626 16232 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:20:22.518002 16232 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1329b85e557d4f5282015dc59efffd30: Generated new TSK 0
I20260809 02:20:22.518743 16232 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:20:22.525316 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 16124
W20260809 02:20:22.539911 16100 connection.cc:535] client connection to 127.15.75.190:37653 recv error: Network error: failed to read from TLS socket (remote: 127.15.75.190:37653): Connection reset by peer (error 104)
2026-08-09T02:20:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1646 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241741788589-15662-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:20:22 dist-test-slave-s11t krb5kdc[16267](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:20:22 dist-test-slave-s11t krb5kdc[16267](info): set up 2 sockets
Aug 09 02:20:22 dist-test-slave-s11t krb5kdc[16267](info): commencing operation
krb5kdc: starting...
W20260809 02:20:23.749912 15662 mini_kdc.cc:121] Time spent starting KDC: real 1.196s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Aug 09 02:20:23 dist-test-slave-s11t krb5kdc[16267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786242023, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:20:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:20:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.75.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab.
Entry for principal kudu/127.15.75.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab.
I20260809 02:20:23.871495 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241741788589-15662-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241741788589-15662-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241741788589-15662-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:35747
--webserver_interface=127.15.75.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab
--principal=kudu/127.15.75.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:36493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.75.190:35747
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:20:23.980890 16284 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:20:23.981151 16284 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:20:23.981191 16284 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:20:23.985181 16284 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:20:23.985232 16284 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:20:23.985251 16284 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:20:23.985292 16284 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:20:23.985327 16284 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:20:23.990651 16284 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:36493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241741788589-15662-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241741788589-15662-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.75.190:35747
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab
--principal=kudu/127.15.75.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:35747
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.75.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16284
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241741788589-15662-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:20:23.991667 16284 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:20:23.992660 16284 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:20:24.001091 16290 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:20:24.001158 16291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:20:24.001098 16295 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:20:24.001420 16284 server_base.cc:1034] running on GCE node
I20260809 02:20:24.001963 16284 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:20:24.003031 16284 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:20:24.004217 16284 hybrid_clock.cc:648] HybridClock initialized: now 1786242024004202 us; error 35 us; skew 500 ppm
Aug 09 02:20:24 dist-test-slave-s11t krb5kdc[16267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786242024, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:20:24.007097 16284 init.cc:377] Logged in from keytab as kudu/127.15.75.190@KRBTEST.COM (short username kudu)
I20260809 02:20:24.007954 16284 webserver.cc:460] Webserver started at http://127.15.75.190:44495/ using document root <none> and password file <none>
I20260809 02:20:24.008503 16284 fs_manager.cc:359] Metadata directory not provided
I20260809 02:20:24.008594 16284 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:20:24.008865 16284 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:20:24.011193 16284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241741788589-15662-0/minicluster-data/master-0/data/instance:
uuid: "873ad3b81c914626bb349e668bba78b0"
format_stamp: "Formatted at 2026-08-09 02:20:24 on dist-test-slave-s11t"
server_key: "26ded6c066d2c5ca400a51d87fb70f98"
server_key_iv: "7607e5ac34fc643dfede4fd82a3f4ceb"
server_key_version: "encryptionkey@0"
I20260809 02:20:24.011756 16284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241741788589-15662-0/minicluster-data/master-0/wal/instance:
uuid: "873ad3b81c914626bb349e668bba78b0"
format_stamp: "Formatted at 2026-08-09 02:20:24 on dist-test-slave-s11t"
server_key: "26ded6c066d2c5ca400a51d87fb70f98"
server_key_iv: "7607e5ac34fc643dfede4fd82a3f4ceb"
server_key_version: "encryptionkey@0"
I20260809 02:20:24.015342 16284 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260809 02:20:24.017719 16304 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:20:24.018880 16284 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 02:20:24.019026 16284 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241741788589-15662-0/minicluster-data/master-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241741788589-15662-0/minicluster-data/master-0/wal
uuid: "873ad3b81c914626bb349e668bba78b0"
format_stamp: "Formatted at 2026-08-09 02:20:24 on dist-test-slave-s11t"
server_key: "26ded6c066d2c5ca400a51d87fb70f98"
server_key_iv: "7607e5ac34fc643dfede4fd82a3f4ceb"
server_key_version: "encryptionkey@0"
I20260809 02:20:24.019133 16284 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241741788589-15662-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241741788589-15662-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241741788589-15662-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:20:24.028838 16284 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:20:24.031881 16284 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:20:24.032079 16284 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:20:24.039191 16284 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.190:35747
I20260809 02:20:24.039219 16383 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.190:35747 every 8 connection(s)
I20260809 02:20:24.040071 16284 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
I20260809 02:20:24.043288 16384 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:20:24.047439 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 16284
I20260809 02:20:24.047581 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241741788589-15662-0/minicluster-data/master-0/wal/instance
I20260809 02:20:24.047848 15662 external_mini_cluster.cc:1428] Setting key 0cf4fcea4cf8efe06a207bf2559d25b2
I20260809 02:20:24.049191 16384 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 873ad3b81c914626bb349e668bba78b0: Bootstrap starting.
I20260809 02:20:24.051373 16384 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 873ad3b81c914626bb349e668bba78b0: Neither blocks nor log segments found. Creating new log.
I20260809 02:20:24.052114 16384 log.cc:824] T 00000000000000000000000000000000 P 873ad3b81c914626bb349e668bba78b0: Log is configured to *not* fsync() on all Append() calls
I20260809 02:20:24.054414 16384 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 873ad3b81c914626bb349e668bba78b0: No bootstrap required, opened a new log
I20260809 02:20:24.057507 16388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:24.049257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47287 (local address 127.15.75.190:35747)
0809 02:20:24.049689 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:24.049705 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:24.049739 (+ 34us) server_negotiation.cc:407] Connection header received
0809 02:20:24.050371 (+ 632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:24.050387 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:24.050704 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:24.051038 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:24.051445 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:24.052615 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:24.053859 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:24.054742 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:24.054869 (+ 127us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:24.055820 (+ 951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:20:24.055825 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:20:24.055837 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:20:24.056216 (+ 379us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:20:24.056269 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:20:24.056291 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:20:24.056394 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:24.056609 (+ 215us) server_negotiation.cc:299] Negotiation successful
0809 02:20:24.056799 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":135,"threads_started":1}
I20260809 02:20:24.058626 16384 raft_consensus.cc:348] T 00000000000000000000000000000000 P 873ad3b81c914626bb349e668bba78b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "873ad3b81c914626bb349e668bba78b0" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 35747 } }
I20260809 02:20:24.058882 16384 raft_consensus.cc:374] T 00000000000000000000000000000000 P 873ad3b81c914626bb349e668bba78b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:20:24.058930 16384 raft_consensus.cc:729] T 00000000000000000000000000000000 P 873ad3b81c914626bb349e668bba78b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 873ad3b81c914626bb349e668bba78b0, State: Initialized, Role: FOLLOWER
I20260809 02:20:24.059342 16384 consensus_queue.cc:260] T 00000000000000000000000000000000 P 873ad3b81c914626bb349e668bba78b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "873ad3b81c914626bb349e668bba78b0" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 35747 } }
I20260809 02:20:24.059520 16384 raft_consensus.cc:388] T 00000000000000000000000000000000 P 873ad3b81c914626bb349e668bba78b0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:20:24.059588 16384 raft_consensus.cc:482] T 00000000000000000000000000000000 P 873ad3b81c914626bb349e668bba78b0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:20:24.059695 16384 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 873ad3b81c914626bb349e668bba78b0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:20:24.060446 15662 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:20:24.060802 16384 raft_consensus.cc:504] T 00000000000000000000000000000000 P 873ad3b81c914626bb349e668bba78b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "873ad3b81c914626bb349e668bba78b0" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 35747 } }
I20260809 02:20:24.061172 16384 leader_election.cc:302] T 00000000000000000000000000000000 P 873ad3b81c914626bb349e668bba78b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 873ad3b81c914626bb349e668bba78b0; no voters:
I20260809 02:20:24.061508 16384 leader_election.cc:288] T 00000000000000000000000000000000 P 873ad3b81c914626bb349e668bba78b0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:20:24.061609 16391 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 873ad3b81c914626bb349e668bba78b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:20:24.061857 16391 raft_consensus.cc:686] T 00000000000000000000000000000000 P 873ad3b81c914626bb349e668bba78b0 [term 1 LEADER]: Becoming Leader. State: Replica: 873ad3b81c914626bb349e668bba78b0, State: Running, Role: LEADER
I20260809 02:20:24.062209 16391 consensus_queue.cc:237] T 00000000000000000000000000000000 P 873ad3b81c914626bb349e668bba78b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "873ad3b81c914626bb349e668bba78b0" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 35747 } }
I20260809 02:20:24.062815 16384 sys_catalog.cc:564] T 00000000000000000000000000000000 P 873ad3b81c914626bb349e668bba78b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:20:24.063896 16394 sys_catalog.cc:455] T 00000000000000000000000000000000 P 873ad3b81c914626bb349e668bba78b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "873ad3b81c914626bb349e668bba78b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "873ad3b81c914626bb349e668bba78b0" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 35747 } } }
I20260809 02:20:24.064026 16394 sys_catalog.cc:458] T 00000000000000000000000000000000 P 873ad3b81c914626bb349e668bba78b0 [sys.catalog]: This master's current role is: LEADER
I20260809 02:20:24.064370 16398 sys_catalog.cc:455] T 00000000000000000000000000000000 P 873ad3b81c914626bb349e668bba78b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 873ad3b81c914626bb349e668bba78b0. Latest consensus state: current_term: 1 leader_uuid: "873ad3b81c914626bb349e668bba78b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "873ad3b81c914626bb349e668bba78b0" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 35747 } } }
I20260809 02:20:24.064536 16398 sys_catalog.cc:458] T 00000000000000000000000000000000 P 873ad3b81c914626bb349e668bba78b0 [sys.catalog]: This master's current role is: LEADER
W20260809 02:20:24.068186 16430 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 873ad3b81c914626bb349e668bba78b0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:20:24.068267 16430 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:20:24.068377 16417 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:20:24.068816 16388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:24.063666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40943 (local address 127.15.75.190:35747)
0809 02:20:24.063768 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:24.063772 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:24.064319 (+ 547us) server_negotiation.cc:407] Connection header received
0809 02:20:24.064512 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:24.064515 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:24.064560 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:24.064611 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:24.065076 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:24.065814 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:24.066912 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:24.067238 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:24.067349 (+ 111us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:24.068293 (+ 944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:20:24.068297 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:20:24.068299 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:20:24.068312 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:20:24.068341 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:20:24.068354 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:20:24.068462 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:24.068626 (+ 164us) server_negotiation.cc:299] Negotiation successful
0809 02:20:24.068661 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 02:20:24.069350 16417 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:20:24.074220 16417 catalog_manager.cc:1347] Generated new cluster ID: b224c792ca0f4f8b8e3263bdce881d35
I20260809 02:20:24.074277 16417 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:20:24.090108 16417 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:20:24.091907 16417 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:20:24.100078 16417 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 873ad3b81c914626bb349e668bba78b0: Generated new TSK 0
I20260809 02:20:24.100827 16417 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:20:24.107297 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 16284
W20260809 02:20:24.122675 16265 connection.cc:535] client connection to 127.15.75.190:35747 recv error: Network error: failed to read from TLS socket (remote: 127.15.75.190:35747): Connection reset by peer (error 104)
2026-08-09T02:20:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1582 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241741788589-15662-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:20:24 dist-test-slave-s11t krb5kdc[16437](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:20:24 dist-test-slave-s11t krb5kdc[16437](info): set up 2 sockets
Aug 09 02:20:24 dist-test-slave-s11t krb5kdc[16437](info): commencing operation
krb5kdc: starting...
W20260809 02:20:25.354543 15662 mini_kdc.cc:121] Time spent starting KDC: real 1.217s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Aug 09 02:20:25 dist-test-slave-s11t krb5kdc[16437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786242025, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:20:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:20:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.75.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab.
Entry for principal kudu/127.15.75.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab.
I20260809 02:20:25.477411 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241741788589-15662-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241741788589-15662-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241741788589-15662-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:40129
--webserver_interface=127.15.75.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab
--principal=kudu/127.15.75.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:41097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.75.190:40129
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:20:25.587759 16454 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:20:25.588006 16454 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:20:25.588078 16454 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:20:25.592180 16454 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:20:25.592234 16454 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:20:25.592255 16454 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:20:25.592293 16454 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:20:25.592329 16454 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:20:25.597432 16454 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:41097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241741788589-15662-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241741788589-15662-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.75.190:40129
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab
--principal=kudu/127.15.75.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:40129
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.75.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16454
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241741788589-15662-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:20:25.598362 16454 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:20:25.599331 16454 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:20:25.607824 16463 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:20:25.607867 16454 server_base.cc:1034] running on GCE node
W20260809 02:20:25.607836 16464 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:20:25.607846 16468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:20:25.608753 16454 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:20:25.609876 16454 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:20:25.611043 16454 hybrid_clock.cc:648] HybridClock initialized: now 1786242025611016 us; error 41 us; skew 500 ppm
Aug 09 02:20:25 dist-test-slave-s11t krb5kdc[16437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786242025, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:20:25.613831 16454 init.cc:377] Logged in from keytab as kudu/127.15.75.190@KRBTEST.COM (short username kudu)
I20260809 02:20:25.614734 16454 webserver.cc:460] Webserver started at http://127.15.75.190:36577/ using document root <none> and password file <none>
I20260809 02:20:25.615286 16454 fs_manager.cc:359] Metadata directory not provided
I20260809 02:20:25.615347 16454 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:20:25.615664 16454 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:20:25.617733 16454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241741788589-15662-0/minicluster-data/master-0/data/instance:
uuid: "05f9c337d52b42e682101c686a597570"
format_stamp: "Formatted at 2026-08-09 02:20:25 on dist-test-slave-s11t"
server_key: "b51140a7723b39192664f0e92951c398"
server_key_iv: "5543ec46a7301edf0c8e59ac5038b5f0"
server_key_version: "encryptionkey@0"
I20260809 02:20:25.618327 16454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241741788589-15662-0/minicluster-data/master-0/wal/instance:
uuid: "05f9c337d52b42e682101c686a597570"
format_stamp: "Formatted at 2026-08-09 02:20:25 on dist-test-slave-s11t"
server_key: "b51140a7723b39192664f0e92951c398"
server_key_iv: "5543ec46a7301edf0c8e59ac5038b5f0"
server_key_version: "encryptionkey@0"
I20260809 02:20:25.621922 16454 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 02:20:25.624198 16479 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:20:25.625773 16454 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 02:20:25.625916 16454 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241741788589-15662-0/minicluster-data/master-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241741788589-15662-0/minicluster-data/master-0/wal
uuid: "05f9c337d52b42e682101c686a597570"
format_stamp: "Formatted at 2026-08-09 02:20:25 on dist-test-slave-s11t"
server_key: "b51140a7723b39192664f0e92951c398"
server_key_iv: "5543ec46a7301edf0c8e59ac5038b5f0"
server_key_version: "encryptionkey@0"
I20260809 02:20:25.626021 16454 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241741788589-15662-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241741788589-15662-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241741788589-15662-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:20:25.630420 16454 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:20:25.633198 16454 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:20:25.633347 16454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:20:25.640486 16454 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.190:40129
I20260809 02:20:25.640514 16567 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.190:40129 every 8 connection(s)
I20260809 02:20:25.641667 16454 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
I20260809 02:20:25.642596 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 16454
I20260809 02:20:25.642757 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241741788589-15662-0/minicluster-data/master-0/wal/instance
I20260809 02:20:25.643066 15662 external_mini_cluster.cc:1428] Setting key 9f3b6a8d581113330c4edac3037be9b2
I20260809 02:20:25.645020 16568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:20:25.648173 16575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:25.644445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36965 (local address 127.15.75.190:40129)
0809 02:20:25.644918 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:25.644933 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:25.644967 (+ 34us) server_negotiation.cc:407] Connection header received
0809 02:20:25.645645 (+ 678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:25.645662 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:25.645929 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:25.646258 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:25.647061 (+ 803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:20:25.647065 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:20:25.647077 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:20:25.647088 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:20:25.647139 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:20:25.647158 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:20:25.647304 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:25.647377 (+ 73us) server_negotiation.cc:299] Negotiation successful
0809 02:20:25.647539 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":163,"threads_started":1}
I20260809 02:20:25.651048 15662 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:20:25.651295 16568 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05f9c337d52b42e682101c686a597570: Bootstrap starting.
I20260809 02:20:25.653717 16568 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 05f9c337d52b42e682101c686a597570: Neither blocks nor log segments found. Creating new log.
I20260809 02:20:25.654557 16568 log.cc:824] T 00000000000000000000000000000000 P 05f9c337d52b42e682101c686a597570: Log is configured to *not* fsync() on all Append() calls
I20260809 02:20:25.654781 16575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:25.653108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39001 (local address 127.15.75.190:40129)
0809 02:20:25.653223 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:25.653228 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:25.653405 (+ 177us) server_negotiation.cc:407] Connection header received
0809 02:20:25.653572 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:25.653575 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:25.653604 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:25.653683 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:25.654377 (+ 694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:20:25.654380 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:20:25.654383 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:20:25.654394 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:20:25.654427 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:20:25.654438 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:20:25.654527 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:25.654618 (+ 91us) server_negotiation.cc:299] Negotiation successful
0809 02:20:25.654656 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 02:20:25.656781 16568 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05f9c337d52b42e682101c686a597570: No bootstrap required, opened a new log
I20260809 02:20:25.660578 16568 raft_consensus.cc:348] T 00000000000000000000000000000000 P 05f9c337d52b42e682101c686a597570 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05f9c337d52b42e682101c686a597570" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 40129 } }
I20260809 02:20:25.660893 16568 raft_consensus.cc:374] T 00000000000000000000000000000000 P 05f9c337d52b42e682101c686a597570 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:20:25.660957 16568 raft_consensus.cc:729] T 00000000000000000000000000000000 P 05f9c337d52b42e682101c686a597570 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05f9c337d52b42e682101c686a597570, State: Initialized, Role: FOLLOWER
I20260809 02:20:25.661345 16568 consensus_queue.cc:260] T 00000000000000000000000000000000 P 05f9c337d52b42e682101c686a597570 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05f9c337d52b42e682101c686a597570" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 40129 } }
I20260809 02:20:25.661500 16568 raft_consensus.cc:388] T 00000000000000000000000000000000 P 05f9c337d52b42e682101c686a597570 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:20:25.661569 16568 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05f9c337d52b42e682101c686a597570 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:20:25.661670 16568 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 05f9c337d52b42e682101c686a597570 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:20:25.662521 16568 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05f9c337d52b42e682101c686a597570 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05f9c337d52b42e682101c686a597570" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 40129 } }
I20260809 02:20:25.662878 16568 leader_election.cc:302] T 00000000000000000000000000000000 P 05f9c337d52b42e682101c686a597570 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 05f9c337d52b42e682101c686a597570; no voters:
I20260809 02:20:25.663173 16568 leader_election.cc:288] T 00000000000000000000000000000000 P 05f9c337d52b42e682101c686a597570 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:20:25.663304 16584 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 05f9c337d52b42e682101c686a597570 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:20:25.663540 16584 raft_consensus.cc:686] T 00000000000000000000000000000000 P 05f9c337d52b42e682101c686a597570 [term 1 LEADER]: Becoming Leader. State: Replica: 05f9c337d52b42e682101c686a597570, State: Running, Role: LEADER
I20260809 02:20:25.663880 16584 consensus_queue.cc:237] T 00000000000000000000000000000000 P 05f9c337d52b42e682101c686a597570 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05f9c337d52b42e682101c686a597570" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 40129 } }
I20260809 02:20:25.664480 16568 sys_catalog.cc:564] T 00000000000000000000000000000000 P 05f9c337d52b42e682101c686a597570 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:20:25.665735 16587 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05f9c337d52b42e682101c686a597570 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "05f9c337d52b42e682101c686a597570" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05f9c337d52b42e682101c686a597570" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 40129 } } }
I20260809 02:20:25.665755 16589 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05f9c337d52b42e682101c686a597570 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 05f9c337d52b42e682101c686a597570. Latest consensus state: current_term: 1 leader_uuid: "05f9c337d52b42e682101c686a597570" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05f9c337d52b42e682101c686a597570" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 40129 } } }
I20260809 02:20:25.665932 16587 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05f9c337d52b42e682101c686a597570 [sys.catalog]: This master's current role is: LEADER
I20260809 02:20:25.665951 16589 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05f9c337d52b42e682101c686a597570 [sys.catalog]: This master's current role is: LEADER
W20260809 02:20:25.669430 16615 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 05f9c337d52b42e682101c686a597570: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:20:25.669495 16615 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:20:25.669641 16597 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:20:25.670542 16597 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:20:25.675576 16597 catalog_manager.cc:1347] Generated new cluster ID: da3fd16724934336b707beeb70d604b3
I20260809 02:20:25.675638 16597 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:20:25.699470 16597 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:20:25.699734 16597 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:20:25.715165 16597 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 05f9c337d52b42e682101c686a597570: Generated new TSK 0
I20260809 02:20:25.715788 16597 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:20:25.722831 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 16454
2026-08-09T02:20:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1610 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241741788589-15662-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:20:25 dist-test-slave-s11t krb5kdc[16620](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:20:25 dist-test-slave-s11t krb5kdc[16620](info): set up 2 sockets
Aug 09 02:20:25 dist-test-slave-s11t krb5kdc[16620](info): commencing operation
krb5kdc: starting...
W20260809 02:20:26.955432 15662 mini_kdc.cc:121] Time spent starting KDC: real 1.208s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Aug 09 02:20:27 dist-test-slave-s11t krb5kdc[16620](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786242027, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:20:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:20:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.75.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab.
Entry for principal kudu/127.15.75.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab.
I20260809 02:20:27.079133 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241741788589-15662-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241741788589-15662-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241741788589-15662-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:46291
--webserver_interface=127.15.75.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab
--principal=kudu/127.15.75.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:45579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.75.190:46291
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:20:27.189378 16644 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:20:27.189656 16644 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:20:27.189735 16644 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:20:27.193655 16644 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:20:27.193747 16644 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:20:27.193796 16644 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:20:27.193830 16644 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:20:27.193881 16644 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:20:27.199005 16644 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:45579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241741788589-15662-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241741788589-15662-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.75.190:46291
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab
--principal=kudu/127.15.75.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:46291
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.75.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16644
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241741788589-15662-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:20:27.199995 16644 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:20:27.201009 16644 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:20:27.209419 16650 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:20:27.209453 16644 server_base.cc:1034] running on GCE node
W20260809 02:20:27.209425 16651 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:20:27.209491 16653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:20:27.210220 16644 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:20:27.211236 16644 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:20:27.212428 16644 hybrid_clock.cc:648] HybridClock initialized: now 1786242027212416 us; error 25 us; skew 500 ppm
Aug 09 02:20:27 dist-test-slave-s11t krb5kdc[16620](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786242027, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:20:27.218530 16644 init.cc:377] Logged in from keytab as kudu/127.15.75.190@KRBTEST.COM (short username kudu)
I20260809 02:20:27.219341 16644 webserver.cc:460] Webserver started at http://127.15.75.190:33493/ using document root <none> and password file <none>
I20260809 02:20:27.219834 16644 fs_manager.cc:359] Metadata directory not provided
I20260809 02:20:27.219894 16644 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:20:27.220048 16644 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:20:27.222021 16644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241741788589-15662-0/minicluster-data/master-0/data/instance:
uuid: "e5a410fcdd924f5496b07699b8a3cc1d"
format_stamp: "Formatted at 2026-08-09 02:20:27 on dist-test-slave-s11t"
server_key: "be0c7bfac9565dfa5a5c6ca86915435d"
server_key_iv: "5eaeea50665a80111c1c2fe52864ee74"
server_key_version: "encryptionkey@0"
I20260809 02:20:27.222532 16644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241741788589-15662-0/minicluster-data/master-0/wal/instance:
uuid: "e5a410fcdd924f5496b07699b8a3cc1d"
format_stamp: "Formatted at 2026-08-09 02:20:27 on dist-test-slave-s11t"
server_key: "be0c7bfac9565dfa5a5c6ca86915435d"
server_key_iv: "5eaeea50665a80111c1c2fe52864ee74"
server_key_version: "encryptionkey@0"
I20260809 02:20:27.225979 16644 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 02:20:27.228196 16665 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:20:27.229404 16644 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 02:20:27.229527 16644 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241741788589-15662-0/minicluster-data/master-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241741788589-15662-0/minicluster-data/master-0/wal
uuid: "e5a410fcdd924f5496b07699b8a3cc1d"
format_stamp: "Formatted at 2026-08-09 02:20:27 on dist-test-slave-s11t"
server_key: "be0c7bfac9565dfa5a5c6ca86915435d"
server_key_iv: "5eaeea50665a80111c1c2fe52864ee74"
server_key_version: "encryptionkey@0"
I20260809 02:20:27.229624 16644 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241741788589-15662-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241741788589-15662-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241741788589-15662-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:20:27.247991 16644 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:20:27.251093 16644 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:20:27.251287 16644 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:20:27.258423 16644 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.190:46291
I20260809 02:20:27.258448 16737 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.190:46291 every 8 connection(s)
I20260809 02:20:27.259289 16644 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
I20260809 02:20:27.261994 16739 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:20:27.265199 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 16644
I20260809 02:20:27.265357 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241741788589-15662-0/minicluster-data/master-0/wal/instance
I20260809 02:20:27.265633 15662 external_mini_cluster.cc:1428] Setting key 942651d0e37c77d070764682433f6977
I20260809 02:20:27.267786 16739 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5a410fcdd924f5496b07699b8a3cc1d: Bootstrap starting.
I20260809 02:20:27.270504 16739 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e5a410fcdd924f5496b07699b8a3cc1d: Neither blocks nor log segments found. Creating new log.
I20260809 02:20:27.271371 16739 log.cc:824] T 00000000000000000000000000000000 P e5a410fcdd924f5496b07699b8a3cc1d: Log is configured to *not* fsync() on all Append() calls
I20260809 02:20:27.273500 16739 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5a410fcdd924f5496b07699b8a3cc1d: No bootstrap required, opened a new log
Aug 09 02:20:27 dist-test-slave-s11t krb5kdc[16620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786242027, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:20:27.277598 16739 raft_consensus.cc:348] T 00000000000000000000000000000000 P e5a410fcdd924f5496b07699b8a3cc1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5a410fcdd924f5496b07699b8a3cc1d" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 46291 } }
I20260809 02:20:27.277872 16739 raft_consensus.cc:374] T 00000000000000000000000000000000 P e5a410fcdd924f5496b07699b8a3cc1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:20:27.277946 16739 raft_consensus.cc:729] T 00000000000000000000000000000000 P e5a410fcdd924f5496b07699b8a3cc1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5a410fcdd924f5496b07699b8a3cc1d, State: Initialized, Role: FOLLOWER
I20260809 02:20:27.278432 16739 consensus_queue.cc:260] T 00000000000000000000000000000000 P e5a410fcdd924f5496b07699b8a3cc1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5a410fcdd924f5496b07699b8a3cc1d" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 46291 } }
I20260809 02:20:27.278607 16739 raft_consensus.cc:388] T 00000000000000000000000000000000 P e5a410fcdd924f5496b07699b8a3cc1d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:20:27.278678 16739 raft_consensus.cc:482] T 00000000000000000000000000000000 P e5a410fcdd924f5496b07699b8a3cc1d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:20:27.278780 16739 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e5a410fcdd924f5496b07699b8a3cc1d [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:20:27.279659 16739 raft_consensus.cc:504] T 00000000000000000000000000000000 P e5a410fcdd924f5496b07699b8a3cc1d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5a410fcdd924f5496b07699b8a3cc1d" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 46291 } }
I20260809 02:20:27.280018 16739 leader_election.cc:302] T 00000000000000000000000000000000 P e5a410fcdd924f5496b07699b8a3cc1d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e5a410fcdd924f5496b07699b8a3cc1d; no voters:
I20260809 02:20:27.280324 16739 leader_election.cc:288] T 00000000000000000000000000000000 P e5a410fcdd924f5496b07699b8a3cc1d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:20:27.280429 16747 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e5a410fcdd924f5496b07699b8a3cc1d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:20:27.280634 16747 raft_consensus.cc:686] T 00000000000000000000000000000000 P e5a410fcdd924f5496b07699b8a3cc1d [term 1 LEADER]: Becoming Leader. State: Replica: e5a410fcdd924f5496b07699b8a3cc1d, State: Running, Role: LEADER
I20260809 02:20:27.280763 16742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:27.267006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55891 (local address 127.15.75.190:46291)
0809 02:20:27.267425 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:27.267449 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:27.267492 (+ 43us) server_negotiation.cc:407] Connection header received
0809 02:20:27.268297 (+ 805us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:27.268321 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:27.268759 (+ 438us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:27.269132 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:27.269799 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:27.271304 (+ 1505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:27.272385 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:27.273094 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:27.273261 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:27.275637 (+ 2376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:20:27.275644 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:20:27.275663 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:20:27.275702 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:20:27.277687 (+ 1985us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:27.278178 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:27.278184 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:27.278189 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:27.278325 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:27.278623 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:27.278626 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:27.278628 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:27.279012 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:20:27.279229 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:27.279520 (+ 291us) server_negotiation.cc:299] Negotiation successful
0809 02:20:27.279825 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":100,"threads_started":1}
I20260809 02:20:27.281029 16747 consensus_queue.cc:237] T 00000000000000000000000000000000 P e5a410fcdd924f5496b07699b8a3cc1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5a410fcdd924f5496b07699b8a3cc1d" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 46291 } }
I20260809 02:20:27.281584 16739 sys_catalog.cc:564] T 00000000000000000000000000000000 P e5a410fcdd924f5496b07699b8a3cc1d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:20:27.283969 15662 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:20:27.284327 16749 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5a410fcdd924f5496b07699b8a3cc1d [sys.catalog]: SysCatalogTable state changed. Reason: New leader e5a410fcdd924f5496b07699b8a3cc1d. Latest consensus state: current_term: 1 leader_uuid: "e5a410fcdd924f5496b07699b8a3cc1d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5a410fcdd924f5496b07699b8a3cc1d" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 46291 } } }
I20260809 02:20:27.284325 16748 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5a410fcdd924f5496b07699b8a3cc1d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e5a410fcdd924f5496b07699b8a3cc1d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5a410fcdd924f5496b07699b8a3cc1d" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 46291 } } }
I20260809 02:20:27.284507 16749 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5a410fcdd924f5496b07699b8a3cc1d [sys.catalog]: This master's current role is: LEADER
I20260809 02:20:27.284507 16748 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5a410fcdd924f5496b07699b8a3cc1d [sys.catalog]: This master's current role is: LEADER
W20260809 02:20:27.287577 16771 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e5a410fcdd924f5496b07699b8a3cc1d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:20:27.287674 16771 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:20:27.287778 16772 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:20:27.288477 16772 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:20:27.294786 16772 catalog_manager.cc:1347] Generated new cluster ID: b7b59d4ffefc4e09923f39d9cd073ca0
I20260809 02:20:27.294872 16772 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:20:27.294911 16742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:27.286218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41085 (local address 127.15.75.190:46291)
0809 02:20:27.286317 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:27.286321 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:27.286615 (+ 294us) server_negotiation.cc:407] Connection header received
0809 02:20:27.286719 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:27.286722 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:27.286766 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:27.286848 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:27.287640 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:27.288437 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:27.289559 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:27.289928 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:27.290040 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:27.291370 (+ 1330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:20:27.291374 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:20:27.291376 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:20:27.291412 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:20:27.292986 (+ 1574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:27.293499 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:27.293502 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:27.293504 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:27.293593 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:27.293903 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:27.293906 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:27.293908 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:27.294114 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:20:27.294238 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:27.294593 (+ 355us) server_negotiation.cc:299] Negotiation successful
0809 02:20:27.294708 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 02:20:27.304032 16772 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:20:27.305095 16772 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:20:27.315227 16772 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e5a410fcdd924f5496b07699b8a3cc1d: Generated new TSK 0
I20260809 02:20:27.315943 16772 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:20:27.321573 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 16644
2026-08-09T02:20:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1603 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241741788589-15662-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:20:27 dist-test-slave-s11t krb5kdc[16790](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:20:27 dist-test-slave-s11t krb5kdc[16790](info): set up 2 sockets
Aug 09 02:20:27 dist-test-slave-s11t krb5kdc[16790](info): commencing operation
krb5kdc: starting...
W20260809 02:20:28.562233 15662 mini_kdc.cc:121] Time spent starting KDC: real 1.211s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Aug 09 02:20:28 dist-test-slave-s11t krb5kdc[16790](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786242028, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:20:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:20:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.75.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab.
Entry for principal kudu/127.15.75.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab.
I20260809 02:20:28.685220 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241741788589-15662-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241741788589-15662-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241741788589-15662-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:41403
--webserver_interface=127.15.75.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab
--principal=kudu/127.15.75.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:38217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.75.190:41403
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:20:28.795639 16811 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:20:28.795857 16811 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:20:28.795898 16811 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:20:28.800047 16811 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:20:28.800099 16811 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:20:28.800118 16811 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:20:28.800156 16811 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:20:28.800189 16811 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:20:28.805249 16811 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:38217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241741788589-15662-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241741788589-15662-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.75.190:41403
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab
--principal=kudu/127.15.75.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:41403
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.75.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16811
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241741788589-15662-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:20:28.806169 16811 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:20:28.807125 16811 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:20:28.815708 16826 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:20:28.815820 16819 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:20:28.815773 16821 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:20:28.815755 16811 server_base.cc:1034] running on GCE node
I20260809 02:20:28.816475 16811 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:20:28.817559 16811 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:20:28.818753 16811 hybrid_clock.cc:648] HybridClock initialized: now 1786242028818733 us; error 31 us; skew 500 ppm
Aug 09 02:20:28 dist-test-slave-s11t krb5kdc[16790](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786242028, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:20:28.821398 16811 init.cc:377] Logged in from keytab as kudu/127.15.75.190@KRBTEST.COM (short username kudu)
I20260809 02:20:28.822216 16811 webserver.cc:460] Webserver started at http://127.15.75.190:36719/ using document root <none> and password file <none>
I20260809 02:20:28.822760 16811 fs_manager.cc:359] Metadata directory not provided
I20260809 02:20:28.822852 16811 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:20:28.823091 16811 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:20:28.825119 16811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241741788589-15662-0/minicluster-data/master-0/data/instance:
uuid: "d7ec25201ca040b89ce7025669d0791c"
format_stamp: "Formatted at 2026-08-09 02:20:28 on dist-test-slave-s11t"
server_key: "02284106cffe01d6dc1d59e0833ff63a"
server_key_iv: "93529e2216ec1b35ed42b33bf202bdda"
server_key_version: "encryptionkey@0"
I20260809 02:20:28.825671 16811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241741788589-15662-0/minicluster-data/master-0/wal/instance:
uuid: "d7ec25201ca040b89ce7025669d0791c"
format_stamp: "Formatted at 2026-08-09 02:20:28 on dist-test-slave-s11t"
server_key: "02284106cffe01d6dc1d59e0833ff63a"
server_key_iv: "93529e2216ec1b35ed42b33bf202bdda"
server_key_version: "encryptionkey@0"
I20260809 02:20:28.829190 16811 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 02:20:28.831605 16836 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:20:28.833006 16811 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 02:20:28.833165 16811 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241741788589-15662-0/minicluster-data/master-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241741788589-15662-0/minicluster-data/master-0/wal
uuid: "d7ec25201ca040b89ce7025669d0791c"
format_stamp: "Formatted at 2026-08-09 02:20:28 on dist-test-slave-s11t"
server_key: "02284106cffe01d6dc1d59e0833ff63a"
server_key_iv: "93529e2216ec1b35ed42b33bf202bdda"
server_key_version: "encryptionkey@0"
I20260809 02:20:28.833273 16811 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241741788589-15662-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241741788589-15662-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241741788589-15662-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:20:28.873416 16811 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:20:28.876194 16811 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:20:28.876412 16811 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:20:28.883248 16811 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.190:41403
I20260809 02:20:28.883280 16905 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.190:41403 every 8 connection(s)
I20260809 02:20:28.884160 16811 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
I20260809 02:20:28.886798 16907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:20:28.891340 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 16811
I20260809 02:20:28.891502 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241741788589-15662-0/minicluster-data/master-0/wal/instance
I20260809 02:20:28.891795 15662 external_mini_cluster.cc:1428] Setting key 28026b2ce5d42bfcf63773caa915dc10
I20260809 02:20:28.892418 16907 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7ec25201ca040b89ce7025669d0791c: Bootstrap starting.
I20260809 02:20:28.894728 16907 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d7ec25201ca040b89ce7025669d0791c: Neither blocks nor log segments found. Creating new log.
I20260809 02:20:28.895450 16907 log.cc:824] T 00000000000000000000000000000000 P d7ec25201ca040b89ce7025669d0791c: Log is configured to *not* fsync() on all Append() calls
I20260809 02:20:28.897275 16907 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7ec25201ca040b89ce7025669d0791c: No bootstrap required, opened a new log
I20260809 02:20:28.900491 16910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:28.893155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56497 (local address 127.15.75.190:41403)
0809 02:20:28.893563 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:28.893577 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:28.893611 (+ 34us) server_negotiation.cc:407] Connection header received
0809 02:20:28.894206 (+ 595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:28.894222 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:28.894541 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:28.894861 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:28.895272 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:28.896266 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:28.897249 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:28.897765 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:28.897975 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:28.898899 (+ 924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:20:28.898905 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:20:28.898919 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:20:28.899273 (+ 354us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:20:28.899322 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:20:28.899343 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:20:28.899428 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:28.899592 (+ 164us) server_negotiation.cc:299] Negotiation successful
0809 02:20:28.899812 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":114,"threads_started":1}
I20260809 02:20:28.901311 16907 raft_consensus.cc:348] T 00000000000000000000000000000000 P d7ec25201ca040b89ce7025669d0791c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7ec25201ca040b89ce7025669d0791c" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 41403 } }
I20260809 02:20:28.901546 16907 raft_consensus.cc:374] T 00000000000000000000000000000000 P d7ec25201ca040b89ce7025669d0791c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:20:28.901628 16907 raft_consensus.cc:729] T 00000000000000000000000000000000 P d7ec25201ca040b89ce7025669d0791c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7ec25201ca040b89ce7025669d0791c, State: Initialized, Role: FOLLOWER
I20260809 02:20:28.902019 16907 consensus_queue.cc:260] T 00000000000000000000000000000000 P d7ec25201ca040b89ce7025669d0791c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7ec25201ca040b89ce7025669d0791c" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 41403 } }
I20260809 02:20:28.902189 16907 raft_consensus.cc:388] T 00000000000000000000000000000000 P d7ec25201ca040b89ce7025669d0791c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:20:28.902282 16907 raft_consensus.cc:482] T 00000000000000000000000000000000 P d7ec25201ca040b89ce7025669d0791c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:20:28.902427 16907 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d7ec25201ca040b89ce7025669d0791c [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:20:28.903270 15662 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:20:28.903348 16907 raft_consensus.cc:504] T 00000000000000000000000000000000 P d7ec25201ca040b89ce7025669d0791c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7ec25201ca040b89ce7025669d0791c" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 41403 } }
I20260809 02:20:28.903712 16907 leader_election.cc:302] T 00000000000000000000000000000000 P d7ec25201ca040b89ce7025669d0791c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d7ec25201ca040b89ce7025669d0791c; no voters:
I20260809 02:20:28.904026 16907 leader_election.cc:288] T 00000000000000000000000000000000 P d7ec25201ca040b89ce7025669d0791c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:20:28.904089 16913 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d7ec25201ca040b89ce7025669d0791c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:20:28.904286 16913 raft_consensus.cc:686] T 00000000000000000000000000000000 P d7ec25201ca040b89ce7025669d0791c [term 1 LEADER]: Becoming Leader. State: Replica: d7ec25201ca040b89ce7025669d0791c, State: Running, Role: LEADER
I20260809 02:20:28.904631 16913 consensus_queue.cc:237] T 00000000000000000000000000000000 P d7ec25201ca040b89ce7025669d0791c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7ec25201ca040b89ce7025669d0791c" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 41403 } }
I20260809 02:20:28.905284 16907 sys_catalog.cc:564] T 00000000000000000000000000000000 P d7ec25201ca040b89ce7025669d0791c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:20:28.906279 16915 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7ec25201ca040b89ce7025669d0791c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d7ec25201ca040b89ce7025669d0791c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7ec25201ca040b89ce7025669d0791c" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 41403 } } }
I20260809 02:20:28.906688 16915 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7ec25201ca040b89ce7025669d0791c [sys.catalog]: This master's current role is: LEADER
I20260809 02:20:28.906581 16914 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7ec25201ca040b89ce7025669d0791c [sys.catalog]: SysCatalogTable state changed. Reason: New leader d7ec25201ca040b89ce7025669d0791c. Latest consensus state: current_term: 1 leader_uuid: "d7ec25201ca040b89ce7025669d0791c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7ec25201ca040b89ce7025669d0791c" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 41403 } } }
I20260809 02:20:28.907558 16914 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7ec25201ca040b89ce7025669d0791c [sys.catalog]: This master's current role is: LEADER
I20260809 02:20:28.907878 16934 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:20:28.910087 16910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:28.905399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43303 (local address 127.15.75.190:41403)
0809 02:20:28.905530 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:28.905533 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:28.905874 (+ 341us) server_negotiation.cc:407] Connection header received
0809 02:20:28.906060 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:28.906063 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:28.906107 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:28.906160 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:28.906604 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:28.907309 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:28.908436 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:28.908748 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:28.908860 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:28.909648 (+ 788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:20:28.909651 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:20:28.909654 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:20:28.909666 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:20:28.909698 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:20:28.909709 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:20:28.909787 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:28.909878 (+ 91us) server_negotiation.cc:299] Negotiation successful
0809 02:20:28.909932 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 02:20:28.910974 16934 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:20:28.916069 16934 catalog_manager.cc:1347] Generated new cluster ID: 62ffbebc42ba44c7b7d0c56ec90afa69
I20260809 02:20:28.916136 16934 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:20:28.933879 16934 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:20:28.935027 16934 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:20:28.942642 16934 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d7ec25201ca040b89ce7025669d0791c: Generated new TSK 0
I20260809 02:20:28.943328 16934 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:20:28.952569 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 16811
2026-08-09T02:20:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1630 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241741788589-15662-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:20:29 dist-test-slave-s11t krb5kdc[16955](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:20:29 dist-test-slave-s11t krb5kdc[16955](info): set up 2 sockets
Aug 09 02:20:29 dist-test-slave-s11t krb5kdc[16955](info): commencing operation
krb5kdc: starting...
W20260809 02:20:30.189886 15662 mini_kdc.cc:121] Time spent starting KDC: real 1.208s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Aug 09 02:20:30 dist-test-slave-s11t krb5kdc[16955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786242030, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:20:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:20:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.75.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab.
Entry for principal kudu/127.15.75.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab.
I20260809 02:20:30.315294 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241741788589-15662-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241741788589-15662-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241741788589-15662-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:35803
--webserver_interface=127.15.75.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab
--principal=kudu/127.15.75.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:44583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.75.190:35803
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:20:30.427531 16973 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:20:30.427762 16973 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:20:30.427801 16973 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:20:30.431787 16973 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:20:30.431845 16973 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:20:30.431864 16973 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:20:30.431880 16973 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:20:30.431938 16973 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:20:30.437108 16973 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:44583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241741788589-15662-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241741788589-15662-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.75.190:35803
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab
--principal=kudu/127.15.75.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:35803
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.75.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16973
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241741788589-15662-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:20:30.438076 16973 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:20:30.439078 16973 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:20:30.447438 16980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:20:30.447593 16973 server_base.cc:1034] running on GCE node
W20260809 02:20:30.447469 16983 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:20:30.447459 16981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:20:30.448161 16973 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:20:30.449299 16973 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:20:30.450484 16973 hybrid_clock.cc:648] HybridClock initialized: now 1786242030450453 us; error 43 us; skew 500 ppm
Aug 09 02:20:30 dist-test-slave-s11t krb5kdc[16955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786242030, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:20:30.453191 16973 init.cc:377] Logged in from keytab as kudu/127.15.75.190@KRBTEST.COM (short username kudu)
I20260809 02:20:30.454015 16973 webserver.cc:460] Webserver started at http://127.15.75.190:40215/ using document root <none> and password file <none>
I20260809 02:20:30.454548 16973 fs_manager.cc:359] Metadata directory not provided
I20260809 02:20:30.454636 16973 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:20:30.454864 16973 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:20:30.456912 16973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241741788589-15662-0/minicluster-data/master-0/data/instance:
uuid: "0e6e19bd1ecd47e29a155d56233d2abb"
format_stamp: "Formatted at 2026-08-09 02:20:30 on dist-test-slave-s11t"
server_key: "bc317e380b80f5d66e5d98e6c3bff75b"
server_key_iv: "60c37bc262cf20f57d49c0a71bb897c6"
server_key_version: "encryptionkey@0"
I20260809 02:20:30.457474 16973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241741788589-15662-0/minicluster-data/master-0/wal/instance:
uuid: "0e6e19bd1ecd47e29a155d56233d2abb"
format_stamp: "Formatted at 2026-08-09 02:20:30 on dist-test-slave-s11t"
server_key: "bc317e380b80f5d66e5d98e6c3bff75b"
server_key_iv: "60c37bc262cf20f57d49c0a71bb897c6"
server_key_version: "encryptionkey@0"
I20260809 02:20:30.460979 16973 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 02:20:30.463282 16995 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:20:30.464414 16973 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 02:20:30.464552 16973 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241741788589-15662-0/minicluster-data/master-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241741788589-15662-0/minicluster-data/master-0/wal
uuid: "0e6e19bd1ecd47e29a155d56233d2abb"
format_stamp: "Formatted at 2026-08-09 02:20:30 on dist-test-slave-s11t"
server_key: "bc317e380b80f5d66e5d98e6c3bff75b"
server_key_iv: "60c37bc262cf20f57d49c0a71bb897c6"
server_key_version: "encryptionkey@0"
I20260809 02:20:30.464658 16973 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241741788589-15662-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241741788589-15662-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241741788589-15662-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:20:30.469049 16973 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:20:30.471668 16973 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:20:30.471859 16973 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:20:30.478674 16973 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.190:35803
I20260809 02:20:30.478701 17072 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.190:35803 every 8 connection(s)
I20260809 02:20:30.479553 16973 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
I20260809 02:20:30.482307 17073 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:20:30.482398 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 16973
I20260809 02:20:30.482543 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241741788589-15662-0/minicluster-data/master-0/wal/instance
I20260809 02:20:30.482852 15662 external_mini_cluster.cc:1428] Setting key 961b541221aadffc4477b2cce995dd71
I20260809 02:20:30.487881 17077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:30.484183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51355 (local address 127.15.75.190:35803)
0809 02:20:30.484620 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:30.484634 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:30.484667 (+ 33us) server_negotiation.cc:407] Connection header received
0809 02:20:30.485457 (+ 790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:30.485475 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:30.485729 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:30.486043 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:30.486803 (+ 760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:20:30.486808 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:20:30.486828 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:20:30.486840 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:20:30.486890 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:20:30.486909 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:20:30.487057 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:30.487126 (+ 69us) server_negotiation.cc:299] Negotiation successful
0809 02:20:30.487264 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":126,"threads_started":1}
I20260809 02:20:30.488965 17073 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e6e19bd1ecd47e29a155d56233d2abb: Bootstrap starting.
I20260809 02:20:30.490612 15662 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:20:30.491425 17073 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0e6e19bd1ecd47e29a155d56233d2abb: Neither blocks nor log segments found. Creating new log.
I20260809 02:20:30.492183 17073 log.cc:824] T 00000000000000000000000000000000 P 0e6e19bd1ecd47e29a155d56233d2abb: Log is configured to *not* fsync() on all Append() calls
I20260809 02:20:30.494079 17073 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e6e19bd1ecd47e29a155d56233d2abb: No bootstrap required, opened a new log
I20260809 02:20:30.495395 17077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:30.492591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40209 (local address 127.15.75.190:35803)
0809 02:20:30.492733 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:30.492737 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:30.494240 (+ 1503us) server_negotiation.cc:407] Connection header received
0809 02:20:30.494284 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:30.494287 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:30.494316 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:30.494406 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:30.495050 (+ 644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:20:30.495054 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:20:30.495056 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:20:30.495068 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:20:30.495094 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:20:30.495107 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:20:30.495196 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:30.495256 (+ 60us) server_negotiation.cc:299] Negotiation successful
0809 02:20:30.495290 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 02:20:30.498031 17073 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0e6e19bd1ecd47e29a155d56233d2abb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e6e19bd1ecd47e29a155d56233d2abb" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 35803 } }
I20260809 02:20:30.498230 17073 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0e6e19bd1ecd47e29a155d56233d2abb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:20:30.498265 17073 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0e6e19bd1ecd47e29a155d56233d2abb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e6e19bd1ecd47e29a155d56233d2abb, State: Initialized, Role: FOLLOWER
I20260809 02:20:30.498641 17073 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0e6e19bd1ecd47e29a155d56233d2abb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e6e19bd1ecd47e29a155d56233d2abb" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 35803 } }
I20260809 02:20:30.498760 17073 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0e6e19bd1ecd47e29a155d56233d2abb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:20:30.498806 17073 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0e6e19bd1ecd47e29a155d56233d2abb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:20:30.498876 17073 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0e6e19bd1ecd47e29a155d56233d2abb [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:20:30.499679 17073 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0e6e19bd1ecd47e29a155d56233d2abb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e6e19bd1ecd47e29a155d56233d2abb" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 35803 } }
I20260809 02:20:30.500061 17073 leader_election.cc:302] T 00000000000000000000000000000000 P 0e6e19bd1ecd47e29a155d56233d2abb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0e6e19bd1ecd47e29a155d56233d2abb; no voters:
I20260809 02:20:30.500393 17073 leader_election.cc:288] T 00000000000000000000000000000000 P 0e6e19bd1ecd47e29a155d56233d2abb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:20:30.500522 17087 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0e6e19bd1ecd47e29a155d56233d2abb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:20:30.500787 17087 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0e6e19bd1ecd47e29a155d56233d2abb [term 1 LEADER]: Becoming Leader. State: Replica: 0e6e19bd1ecd47e29a155d56233d2abb, State: Running, Role: LEADER
I20260809 02:20:30.501120 17087 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0e6e19bd1ecd47e29a155d56233d2abb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e6e19bd1ecd47e29a155d56233d2abb" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 35803 } }
I20260809 02:20:30.501649 17073 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0e6e19bd1ecd47e29a155d56233d2abb [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:20:30.502960 17089 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e6e19bd1ecd47e29a155d56233d2abb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0e6e19bd1ecd47e29a155d56233d2abb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e6e19bd1ecd47e29a155d56233d2abb" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 35803 } } }
I20260809 02:20:30.502988 17090 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e6e19bd1ecd47e29a155d56233d2abb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0e6e19bd1ecd47e29a155d56233d2abb. Latest consensus state: current_term: 1 leader_uuid: "0e6e19bd1ecd47e29a155d56233d2abb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e6e19bd1ecd47e29a155d56233d2abb" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 35803 } } }
I20260809 02:20:30.503127 17089 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e6e19bd1ecd47e29a155d56233d2abb [sys.catalog]: This master's current role is: LEADER
I20260809 02:20:30.503144 17090 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e6e19bd1ecd47e29a155d56233d2abb [sys.catalog]: This master's current role is: LEADER
I20260809 02:20:30.503974 17098 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:20:30.506615 17098 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:20:30.511523 17098 catalog_manager.cc:1347] Generated new cluster ID: edc4c3b605314dba87491882da427195
I20260809 02:20:30.511584 17098 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:20:30.523269 17098 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:20:30.523572 17098 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:20:30.529021 17098 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0e6e19bd1ecd47e29a155d56233d2abb: Generated new TSK 0
I20260809 02:20:30.529628 17098 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:20:30.540489 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 16973
2026-08-09T02:20:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1588 ms)
[----------] 18 tests from AuthTokenIssuingTest (28856 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241741788589-15662-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:20:30 dist-test-slave-s11t krb5kdc[17121](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:20:30 dist-test-slave-s11t krb5kdc[17121](info): set up 2 sockets
Aug 09 02:20:30 dist-test-slave-s11t krb5kdc[17121](info): commencing operation
krb5kdc: starting...
W20260809 02:20:31.782094 15662 mini_kdc.cc:121] Time spent starting KDC: real 1.212s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Aug 09 02:20:31 dist-test-slave-s11t krb5kdc[17121](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786242031, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:20:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:20:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.75.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab.
Entry for principal kudu/127.15.75.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab.
I20260809 02:20:31.909613 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241741788589-15662-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241741788589-15662-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241741788589-15662-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:39175
--webserver_interface=127.15.75.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab
--principal=kudu/127.15.75.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:46235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.75.190:39175,127.15.75.189:42969,127.15.75.188:43171
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:20:32.033778 17141 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 02:20:32.034029 17141 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:20:32.034050 17141 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:20:32.034144 17141 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:20:32.038033 17141 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:20:32.038077 17141 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:20:32.038096 17141 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:20:32.038157 17141 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:20:32.038187 17141 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:20:32.043246 17141 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:46235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241741788589-15662-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241741788589-15662-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.75.190:39175,127.15.75.189:42969,127.15.75.188:43171
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab
--principal=kudu/127.15.75.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:39175
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.75.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17141
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241741788589-15662-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:20:32.044191 17141 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:20:32.045181 17141 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:20:32.053575 17151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:20:32.053575 17149 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:20:32.053606 17153 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:20:32.053844 17141 server_base.cc:1034] running on GCE node
I20260809 02:20:32.054375 17141 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:20:32.055413 17141 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:20:32.056581 17141 hybrid_clock.cc:648] HybridClock initialized: now 1786242032056563 us; error 29 us; skew 500 ppm
Aug 09 02:20:32 dist-test-slave-s11t krb5kdc[17121](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786242032, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:20:32.059283 17141 init.cc:377] Logged in from keytab as kudu/127.15.75.190@KRBTEST.COM (short username kudu)
I20260809 02:20:32.060141 17141 webserver.cc:460] Webserver started at http://127.15.75.190:46677/ using document root <none> and password file <none>
I20260809 02:20:32.060642 17141 fs_manager.cc:359] Metadata directory not provided
I20260809 02:20:32.060768 17141 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:20:32.060967 17141 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:20:32.062896 17141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241741788589-15662-0/minicluster-data/master-0/data/instance:
uuid: "cdf77404778242bcad0bec1b3345c6cc"
format_stamp: "Formatted at 2026-08-09 02:20:32 on dist-test-slave-s11t"
server_key: "9802311cd9fb9d2168ecfcdfb1785d85"
server_key_iv: "c72266f7323a1303c8c2d12e04c6cf59"
server_key_version: "encryptionkey@0"
I20260809 02:20:32.063401 17141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241741788589-15662-0/minicluster-data/master-0/wal/instance:
uuid: "cdf77404778242bcad0bec1b3345c6cc"
format_stamp: "Formatted at 2026-08-09 02:20:32 on dist-test-slave-s11t"
server_key: "9802311cd9fb9d2168ecfcdfb1785d85"
server_key_iv: "c72266f7323a1303c8c2d12e04c6cf59"
server_key_version: "encryptionkey@0"
I20260809 02:20:32.066957 17141 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260809 02:20:32.069228 17163 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:20:32.070509 17141 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 02:20:32.070648 17141 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241741788589-15662-0/minicluster-data/master-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241741788589-15662-0/minicluster-data/master-0/wal
uuid: "cdf77404778242bcad0bec1b3345c6cc"
format_stamp: "Formatted at 2026-08-09 02:20:32 on dist-test-slave-s11t"
server_key: "9802311cd9fb9d2168ecfcdfb1785d85"
server_key_iv: "c72266f7323a1303c8c2d12e04c6cf59"
server_key_version: "encryptionkey@0"
I20260809 02:20:32.070758 17141 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241741788589-15662-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241741788589-15662-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241741788589-15662-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:20:32.090086 17141 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:20:32.092832 17141 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:20:32.093036 17141 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:20:32.100036 17141 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.190:39175
I20260809 02:20:32.100332 17236 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.190:39175 every 8 connection(s)
I20260809 02:20:32.101055 17141 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
I20260809 02:20:32.103807 17238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:20:32.106933 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 17141
I20260809 02:20:32.107093 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241741788589-15662-0/minicluster-data/master-0/wal/instance
I20260809 02:20:32.107376 15662 external_mini_cluster.cc:1428] Setting key b2281b36f3d1b70b42c6d6f59b5277af
I20260809 02:20:32.107635 17238 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.75.190" port: 39175 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.15.75.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.189@KRBTEST.COM" created.
Aug 09 02:20:32 dist-test-slave-s11t krb5kdc[17121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786242032, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.190@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:20:32.126113 17247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:32.110091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47796 (local address 127.15.75.190:39175)
0809 02:20:32.110671 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:32.110683 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:32.110731 (+ 48us) server_negotiation.cc:407] Connection header received
0809 02:20:32.112105 (+ 1374us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:32.112110 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:32.112277 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:32.112425 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:32.113659 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:32.114522 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:32.116080 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:32.116445 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:32.116562 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:32.121253 (+ 4691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:20:32.121264 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:20:32.121280 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:20:32.121313 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:20:32.123069 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:32.123676 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:32.123681 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:32.123682 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:32.123784 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:32.124144 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:32.124147 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:32.124148 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:32.124502 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:20:32.124605 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:32.125154 (+ 549us) server_negotiation.cc:299] Negotiation successful
0809 02:20:32.125361 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":465,"spinlock_wait_cycles":2432,"thread_start_us":155,"threads_started":1}
I20260809 02:20:32.126116 17243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:32.110090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:39175 (local address 127.0.0.1:47796)
0809 02:20:32.110563 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0809 02:20:32.110613 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 02:20:32.111762 (+ 1149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:20:32.112485 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:20:32.112493 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:20:32.113106 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:20:32.113497 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:20:32.113515 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:20:32.114663 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:20:32.114669 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:20:32.115939 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:20:32.115954 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:20:32.116618 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:20:32.116626 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:20:32.116827 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:32.117396 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:20:32.117417 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:20:32.121089 (+ 3672us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:20:32.123193 (+ 2104us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:20:32.123204 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:20:32.123214 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:20:32.123565 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:20:32.123880 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:20:32.123886 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:20:32.123891 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:20:32.124048 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:20:32.124631 (+ 583us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:20:32.124641 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:20:32.124870 (+ 229us) client_negotiation.cc:769] Sending connection context
0809 02:20:32.125071 (+ 201us) client_negotiation.cc:240] Negotiation successful
0809 02:20:32.125447 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"mutex_wait_us":54,"thread_start_us":147,"threads_started":1}
I20260809 02:20:32.128363 17238 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.75.189" port: 42969 } has no permanent_uuid. Determining permanent_uuid...
I20260809 02:20:32.129115 17243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:32.128885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.189:42969 (local address 127.0.0.1:38150)
0809 02:20:32.128998 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0809 02:20:32.129065 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.189:42969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260809 02:20:32.129424 17168 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.75.189:42969: connect: Connection refused (error 111)
I20260809 02:20:32.130267 17243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:32.130019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.189:42969 (local address 127.0.0.1:38158)
0809 02:20:32.130167 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 02:20:32.130220 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.189:42969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260809 02:20:32.130477 17238 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.75.189:42969: Network error: Client connection negotiation failed: client connection to 127.15.75.189:42969: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.189.keytab.
Entry for principal kudu/127.15.75.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.189.keytab.
I20260809 02:20:32.136803 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241741788589-15662-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241741788589-15662-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241741788589-15662-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241741788589-15662-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.189:42969
--webserver_interface=127.15.75.189
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.189.keytab
--principal=kudu/127.15.75.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:46235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.75.190:39175,127.15.75.189:42969,127.15.75.188:43171
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 02:20:32.179834 17238 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.75.189" port: 42969 } attempt: 1
I20260809 02:20:32.180796 17243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:32.180424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.189:42969 (local address 127.0.0.1:38170)
0809 02:20:32.180577 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 02:20:32.180708 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.189:42969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260809 02:20:32.181514 17243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:32.181297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.189:42969 (local address 127.0.0.1:38178)
0809 02:20:32.181408 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0809 02:20:32.181459 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.189:42969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260809 02:20:32.181646 17238 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.75.189:42969: Network error: Client connection negotiation failed: client connection to 127.15.75.189:42969: connect: Connection refused (error 111)
W20260809 02:20:32.247929 17251 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 02:20:32.248282 17251 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:20:32.248330 17251 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:20:32.248409 17251 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260809 02:20:32.249809 17238 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.75.189" port: 42969 } attempt: 2
I20260809 02:20:32.250855 17243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:32.250422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.189:42969 (local address 127.0.0.1:38190)
0809 02:20:32.250669 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0809 02:20:32.250755 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.189:42969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134}
I20260809 02:20:32.251705 17243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:32.251453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.189:42969 (local address 127.0.0.1:38200)
0809 02:20:32.251581 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0809 02:20:32.251636 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.189:42969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260809 02:20:32.251883 17238 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.75.189:42969: Network error: Client connection negotiation failed: client connection to 127.15.75.189:42969: connect: Connection refused (error 111)
W20260809 02:20:32.253990 17251 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:20:32.254063 17251 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:20:32.254108 17251 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:20:32.254143 17251 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:20:32.254177 17251 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:20:32.259274 17251 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:46235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241741788589-15662-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241741788589-15662-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.15.75.190:39175,127.15.75.189:42969,127.15.75.188:43171
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.189.keytab
--principal=kudu/127.15.75.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.189:42969
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241741788589-15662-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.15.75.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17251
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241741788589-15662-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:20:32.260236 17251 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:20:32.261221 17251 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:20:32.269550 17256 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:20:32.269534 17251 server_base.cc:1034] running on GCE node
W20260809 02:20:32.269526 17259 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:20:32.269554 17257 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:20:32.270284 17251 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:20:32.271303 17251 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:20:32.272464 17251 hybrid_clock.cc:648] HybridClock initialized: now 1786242032272451 us; error 25 us; skew 500 ppm
Aug 09 02:20:32 dist-test-slave-s11t krb5kdc[17121](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786242032, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:20:32.280215 17251 init.cc:377] Logged in from keytab as kudu/127.15.75.189@KRBTEST.COM (short username kudu)
I20260809 02:20:32.281153 17251 webserver.cc:460] Webserver started at http://127.15.75.189:35759/ using document root <none> and password file <none>
I20260809 02:20:32.281700 17251 fs_manager.cc:359] Metadata directory not provided
I20260809 02:20:32.281761 17251 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:20:32.282011 17251 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:20:32.283972 17251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241741788589-15662-0/minicluster-data/master-1/data/instance:
uuid: "531d186bc17e45ea82bdd08bde9d6292"
format_stamp: "Formatted at 2026-08-09 02:20:32 on dist-test-slave-s11t"
server_key: "2454aa4e0a8d9e81de7b1f6b4a7aea0a"
server_key_iv: "1acbc9ee357a3b97b70e2c1edda7b556"
server_key_version: "encryptionkey@0"
I20260809 02:20:32.284521 17251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241741788589-15662-0/minicluster-data/master-1/wal/instance:
uuid: "531d186bc17e45ea82bdd08bde9d6292"
format_stamp: "Formatted at 2026-08-09 02:20:32 on dist-test-slave-s11t"
server_key: "2454aa4e0a8d9e81de7b1f6b4a7aea0a"
server_key_iv: "1acbc9ee357a3b97b70e2c1edda7b556"
server_key_version: "encryptionkey@0"
I20260809 02:20:32.287978 17251 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260809 02:20:32.290293 17270 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:20:32.291356 17251 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 02:20:32.291498 17251 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241741788589-15662-0/minicluster-data/master-1/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241741788589-15662-0/minicluster-data/master-1/wal
uuid: "531d186bc17e45ea82bdd08bde9d6292"
format_stamp: "Formatted at 2026-08-09 02:20:32 on dist-test-slave-s11t"
server_key: "2454aa4e0a8d9e81de7b1f6b4a7aea0a"
server_key_iv: "1acbc9ee357a3b97b70e2c1edda7b556"
server_key_version: "encryptionkey@0"
I20260809 02:20:32.291595 17251 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241741788589-15662-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241741788589-15662-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241741788589-15662-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:20:32.308161 17251 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:20:32.311233 17251 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:20:32.311436 17251 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:20:32.318383 17251 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.189:42969
I20260809 02:20:32.318660 17343 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.189:42969 every 8 connection(s)
I20260809 02:20:32.319311 17251 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241741788589-15662-0/minicluster-data/master-1/data/info.pb
I20260809 02:20:32.322667 17344 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:20:32.322736 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 17251
I20260809 02:20:32.322851 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241741788589-15662-0/minicluster-data/master-1/wal/instance
I20260809 02:20:32.323179 15662 external_mini_cluster.cc:1428] Setting key 0e7e806420a7b4abf45135416050c020
I20260809 02:20:32.327127 17344 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.75.190" port: 39175 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.15.75.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.188@KRBTEST.COM" created.
Aug 09 02:20:32 dist-test-slave-s11t krb5kdc[17121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786242032, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.189@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:20:32.341759 17247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:32.329035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47804 (local address 127.15.75.190:39175)
0809 02:20:32.329169 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:32.329172 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:32.330181 (+ 1009us) server_negotiation.cc:407] Connection header received
0809 02:20:32.331078 (+ 897us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:32.331082 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:32.331146 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:32.331249 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:32.332385 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:32.333037 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:32.334253 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:32.334589 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:32.334707 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:32.337973 (+ 3266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:20:32.337976 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:20:32.337978 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:20:32.338002 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:20:32.339464 (+ 1462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:32.340101 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:32.340103 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:32.340104 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:32.340176 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:32.340584 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:32.340586 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:32.340588 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:32.340811 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:20:32.340890 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:32.341541 (+ 651us) server_negotiation.cc:299] Negotiation successful
0809 02:20:32.341640 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 02:20:32.342902 17349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:32.329526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:39175 (local address 127.0.0.1:47804)
0809 02:20:32.330020 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0809 02:20:32.330077 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 02:20:32.330864 (+ 787us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:20:32.331416 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:20:32.331423 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:20:32.331734 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:20:32.332241 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:20:32.332256 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:20:32.333167 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:20:32.333170 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:20:32.334116 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:20:32.334127 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:20:32.334887 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:20:32.334894 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:20:32.335124 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:32.335741 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:20:32.335767 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:20:32.337799 (+ 2032us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:20:32.339592 (+ 1793us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:20:32.339601 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:20:32.339614 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:20:32.340001 (+ 387us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:20:32.340270 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:20:32.340276 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:20:32.340280 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:20:32.340478 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:20:32.340929 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:20:32.340938 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:20:32.341274 (+ 336us) client_negotiation.cc:769] Sending connection context
0809 02:20:32.341498 (+ 224us) client_negotiation.cc:240] Negotiation successful
0809 02:20:32.341840 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":147,"threads_started":1}
I20260809 02:20:32.343041 17238 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.75.189" port: 42969 } attempt: 3
I20260809 02:20:32.344457 17344 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.75.189" port: 42969 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 02:20:32 dist-test-slave-s11t krb5kdc[17121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786242032, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.190@KRBTEST.COM for kudu/127.15.75.189@KRBTEST.COM
Aug 09 02:20:32 dist-test-slave-s11t krb5kdc[17121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786242032, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.189@KRBTEST.COM for kudu/127.15.75.189@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.188.keytab.
Entry for principal kudu/127.15.75.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.188.keytab.
I20260809 02:20:32.352072 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241741788589-15662-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241741788589-15662-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241741788589-15662-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241741788589-15662-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.188:43171
--webserver_interface=127.15.75.188
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.188.keytab
--principal=kudu/127.15.75.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:46235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.75.190:39175,127.15.75.189:42969,127.15.75.188:43171
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 02:20:32.355759 17243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:32.343446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.189:42969 (local address 127.0.0.1:38216)
0809 02:20:32.343652 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0809 02:20:32.343668 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 02:20:32.343741 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:20:32.344370 (+ 629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:20:32.344373 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:20:32.344549 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:20:32.344655 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:20:32.344662 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:20:32.346013 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:20:32.346018 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:20:32.347372 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:20:32.347381 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:20:32.348137 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:20:32.348141 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:20:32.348249 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:32.348762 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:20:32.348777 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:20:32.350242 (+ 1465us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:20:32.352196 (+ 1954us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:20:32.352202 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:20:32.352207 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:20:32.352490 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:20:32.354470 (+ 1980us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:20:32.354475 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:20:32.354480 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:20:32.354591 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:20:32.355173 (+ 582us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:20:32.355179 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:20:32.355293 (+ 114us) client_negotiation.cc:769] Sending connection context
0809 02:20:32.355425 (+ 132us) client_negotiation.cc:240] Negotiation successful
0809 02:20:32.355570 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117}
I20260809 02:20:32.355985 17352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:32.343528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38216 (local address 127.15.75.189:42969)
0809 02:20:32.343943 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:32.343955 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:32.343978 (+ 23us) server_negotiation.cc:407] Connection header received
0809 02:20:32.344019 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:32.344024 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:32.344225 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:32.344360 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:32.344809 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:32.345883 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:32.347486 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:32.348000 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:32.348126 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:32.350370 (+ 2244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:20:32.350378 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:20:32.350388 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:20:32.350415 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:20:32.352069 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:32.353976 (+ 1907us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:32.353984 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:32.353987 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:32.354090 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:32.354706 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:32.354711 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:32.354714 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:32.355041 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:20:32.355183 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:32.355531 (+ 348us) server_negotiation.cc:299] Negotiation successful
0809 02:20:32.355714 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":47,"server-negotiator.queue_time_us":293,"thread_start_us":99,"threads_started":1}
I20260809 02:20:32.357934 17238 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.75.188" port: 43171 } has no permanent_uuid. Determining permanent_uuid...
I20260809 02:20:32.358700 17243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:32.358426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.188:43171 (local address 127.0.0.1:35462)
0809 02:20:32.358586 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 02:20:32.358638 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.188:43171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260809 02:20:32.358801 17349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:32.344938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.189:42969 (local address 127.0.0.1:38228)
0809 02:20:32.345083 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 02:20:32.345096 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:20:32.345162 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:20:32.345622 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:20:32.345625 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:20:32.345785 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:20:32.345890 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:20:32.345898 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:20:32.347230 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:20:32.347233 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:20:32.347897 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:20:32.347913 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:20:32.348480 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:20:32.348485 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:20:32.348624 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:32.349108 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:20:32.349121 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:20:32.350790 (+ 1669us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:20:32.355959 (+ 5169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:20:32.355963 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:20:32.355965 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:20:32.357226 (+ 1261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:20:32.357541 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:20:32.357546 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:20:32.357550 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:20:32.357655 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:20:32.358149 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:20:32.358153 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:20:32.358262 (+ 109us) client_negotiation.cc:769] Sending connection context
0809 02:20:32.358384 (+ 122us) client_negotiation.cc:240] Negotiation successful
0809 02:20:32.358557 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"mutex_wait_us":70}
I20260809 02:20:32.358829 17354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:32.345033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38228 (local address 127.15.75.189:42969)
0809 02:20:32.345419 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:32.345431 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:32.345442 (+ 11us) server_negotiation.cc:407] Connection header received
0809 02:20:32.345469 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:32.345471 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:32.345501 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:32.345572 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:32.346209 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:32.347104 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:32.348028 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:32.348355 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:32.348477 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:32.350978 (+ 2501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:20:32.350982 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:20:32.350984 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:20:32.351004 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:20:32.355735 (+ 4731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:32.357339 (+ 1604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:32.357344 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:32.357346 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:32.357434 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:32.357770 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:32.357776 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:32.357780 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:32.358006 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:20:32.358109 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:32.358469 (+ 360us) server_negotiation.cc:299] Negotiation successful
0809 02:20:32.358629 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":38,"server-negotiator.queue_time_us":331,"thread_start_us":131,"threads_started":1}
I20260809 02:20:32.359517 17344 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.75.188" port: 43171 } has no permanent_uuid. Determining permanent_uuid...
I20260809 02:20:32.359984 17243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:32.359450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.188:43171 (local address 127.0.0.1:35464)
0809 02:20:32.359846 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0809 02:20:32.359911 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.188:43171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":105,"threads_started":1}
W20260809 02:20:32.360082 17238 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.75.188:43171: Network error: Client connection negotiation failed: client connection to 127.15.75.188:43171: connect: Connection refused (error 111)
I20260809 02:20:32.360227 17349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:32.359909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.188:43171 (local address 127.0.0.1:35474)
0809 02:20:32.360046 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 02:20:32.360139 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.188:43171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260809 02:20:32.360455 17272 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.75.188:43171: connect: Connection refused (error 111)
I20260809 02:20:32.361342 17349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:32.361077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.188:43171 (local address 127.0.0.1:35484)
0809 02:20:32.361209 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 02:20:32.361271 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.188:43171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260809 02:20:32.361526 17344 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.75.188:43171: Network error: Client connection negotiation failed: client connection to 127.15.75.188:43171: connect: Connection refused (error 111)
I20260809 02:20:32.391258 17238 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.75.188" port: 43171 } attempt: 1
I20260809 02:20:32.392072 17243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:32.391726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.188:43171 (local address 127.0.0.1:35500)
0809 02:20:32.391919 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0809 02:20:32.391989 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.188:43171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260809 02:20:32.392843 17243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:32.392594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.188:43171 (local address 127.0.0.1:35506)
0809 02:20:32.392727 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 02:20:32.392780 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.188:43171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260809 02:20:32.393020 17238 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.75.188:43171: Network error: Client connection negotiation failed: client connection to 127.15.75.188:43171: connect: Connection refused (error 111)
I20260809 02:20:32.410774 17344 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.75.188" port: 43171 } attempt: 1
I20260809 02:20:32.411545 17349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:32.411276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.188:43171 (local address 127.0.0.1:35508)
0809 02:20:32.411418 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 02:20:32.411475 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.188:43171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260809 02:20:32.412320 17349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:32.412076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.188:43171 (local address 127.0.0.1:35510)
0809 02:20:32.412217 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 02:20:32.412265 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.188:43171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260809 02:20:32.412537 17344 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.75.188:43171: Network error: Client connection negotiation failed: client connection to 127.15.75.188:43171: connect: Connection refused (error 111)
W20260809 02:20:32.463896 17355 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 02:20:32.464133 17355 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:20:32.464156 17355 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:20:32.464249 17355 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:20:32.468125 17355 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:20:32.468176 17355 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:20:32.468195 17355 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:20:32.468211 17355 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:20:32.468257 17355 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:20:32.468218 17238 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.75.188" port: 43171 } attempt: 2
I20260809 02:20:32.469141 17243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:32.468808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.188:43171 (local address 127.0.0.1:35512)
0809 02:20:32.468976 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 02:20:32.469048 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.188:43171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260809 02:20:32.470031 17243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:32.469790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.188:43171 (local address 127.0.0.1:35520)
0809 02:20:32.469910 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0809 02:20:32.469964 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.188:43171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260809 02:20:32.470199 17238 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.75.188:43171: Network error: Client connection negotiation failed: client connection to 127.15.75.188:43171: connect: Connection refused (error 111)
I20260809 02:20:32.473908 17355 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:46235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241741788589-15662-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241741788589-15662-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.15.75.190:39175,127.15.75.189:42969,127.15.75.188:43171
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.188.keytab
--principal=kudu/127.15.75.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.188:43171
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241741788589-15662-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.15.75.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17355
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241741788589-15662-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:20:32.474900 17355 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:20:32.475864 17355 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 02:20:32.480728 17344 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.75.188" port: 43171 } attempt: 2
I20260809 02:20:32.481449 17349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:32.481188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.188:43171 (local address 127.0.0.1:35530)
0809 02:20:32.481313 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0809 02:20:32.481383 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.188:43171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260809 02:20:32.482167 17349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:32.481956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.188:43171 (local address 127.0.0.1:35534)
0809 02:20:32.482064 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0809 02:20:32.482118 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.188:43171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260809 02:20:32.482290 17344 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.75.188:43171: Network error: Client connection negotiation failed: client connection to 127.15.75.188:43171: connect: Connection refused (error 111)
W20260809 02:20:32.484599 17370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:20:32.484627 17367 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:20:32.484603 17365 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:20:32.485035 17355 server_base.cc:1034] running on GCE node
I20260809 02:20:32.485579 17355 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:20:32.486671 17355 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:20:32.487836 17355 hybrid_clock.cc:648] HybridClock initialized: now 1786242032487817 us; error 28 us; skew 500 ppm
Aug 09 02:20:32 dist-test-slave-s11t krb5kdc[17121](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786242032, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:20:32.490851 17355 init.cc:377] Logged in from keytab as kudu/127.15.75.188@KRBTEST.COM (short username kudu)
I20260809 02:20:32.491722 17355 webserver.cc:460] Webserver started at http://127.15.75.188:33839/ using document root <none> and password file <none>
I20260809 02:20:32.492280 17355 fs_manager.cc:359] Metadata directory not provided
I20260809 02:20:32.492375 17355 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:20:32.492609 17355 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:20:32.494714 17355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241741788589-15662-0/minicluster-data/master-2/data/instance:
uuid: "7da5ecc4ff624a17b8a738e7fdd07eb4"
format_stamp: "Formatted at 2026-08-09 02:20:32 on dist-test-slave-s11t"
server_key: "ac981f8b845cafdfa489edb16de1fcb3"
server_key_iv: "cac5f90304919a15e005acf360287a0c"
server_key_version: "encryptionkey@0"
I20260809 02:20:32.495350 17355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241741788589-15662-0/minicluster-data/master-2/wal/instance:
uuid: "7da5ecc4ff624a17b8a738e7fdd07eb4"
format_stamp: "Formatted at 2026-08-09 02:20:32 on dist-test-slave-s11t"
server_key: "ac981f8b845cafdfa489edb16de1fcb3"
server_key_iv: "cac5f90304919a15e005acf360287a0c"
server_key_version: "encryptionkey@0"
I20260809 02:20:32.499078 17355 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260809 02:20:32.501451 17378 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:20:32.502768 17355 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 02:20:32.502996 17355 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241741788589-15662-0/minicluster-data/master-2/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241741788589-15662-0/minicluster-data/master-2/wal
uuid: "7da5ecc4ff624a17b8a738e7fdd07eb4"
format_stamp: "Formatted at 2026-08-09 02:20:32 on dist-test-slave-s11t"
server_key: "ac981f8b845cafdfa489edb16de1fcb3"
server_key_iv: "cac5f90304919a15e005acf360287a0c"
server_key_version: "encryptionkey@0"
I20260809 02:20:32.503100 17355 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241741788589-15662-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241741788589-15662-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241741788589-15662-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:20:32.527202 17355 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:20:32.530058 17355 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:20:32.530253 17355 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:20:32.537282 17355 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.188:43171
I20260809 02:20:32.537576 17457 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.188:43171 every 8 connection(s)
I20260809 02:20:32.538237 17355 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241741788589-15662-0/minicluster-data/master-2/data/info.pb
I20260809 02:20:32.539062 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 17355
I20260809 02:20:32.539182 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241741788589-15662-0/minicluster-data/master-2/wal/instance
I20260809 02:20:32.539466 15662 external_mini_cluster.cc:1428] Setting key 86b235a1ae7685f58ea3c79b47cbd699
I20260809 02:20:32.541399 17458 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:20:32.545868 17458 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.75.190" port: 39175 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 02:20:32 dist-test-slave-s11t krb5kdc[17121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786242031, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.75.188@KRBTEST.COM
Aug 09 02:20:32 dist-test-slave-s11t krb5kdc[17121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786242032, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.188@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:20:32.554731 17461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:32.540774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35536 (local address 127.15.75.188:43171)
0809 02:20:32.541157 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:32.541177 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:32.541217 (+ 40us) server_negotiation.cc:407] Connection header received
0809 02:20:32.542053 (+ 836us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:32.542075 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:32.542472 (+ 397us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:32.542829 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:32.543514 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:32.545051 (+ 1537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:32.545950 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:32.546627 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:32.546783 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:32.549635 (+ 2852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:20:32.549641 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:20:32.549652 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:20:32.549681 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:20:32.551501 (+ 1820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:32.552087 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:32.552095 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:32.552102 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:32.552250 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:32.552580 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:32.552583 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:32.552584 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:32.553023 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:20:32.553226 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:32.553539 (+ 313us) server_negotiation.cc:299] Negotiation successful
0809 02:20:32.553802 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":244,"thread_start_us":101,"threads_started":1}
I20260809 02:20:32.557593 17247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:32.547784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47808 (local address 127.15.75.190:39175)
0809 02:20:32.547943 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:32.547948 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:32.548341 (+ 393us) server_negotiation.cc:407] Connection header received
0809 02:20:32.548503 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:32.548505 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:32.548547 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:32.548639 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:32.549391 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:32.550099 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:32.551089 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:32.551464 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:32.551587 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:32.554098 (+ 2511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:20:32.554102 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:20:32.554105 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:20:32.554131 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:20:32.555700 (+ 1569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:32.556142 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:32.556145 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:32.556147 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:32.556236 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:32.556551 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:32.556555 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:32.556556 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:32.556822 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:20:32.556932 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:32.557362 (+ 430us) server_negotiation.cc:299] Negotiation successful
0809 02:20:32.557461 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 02:20:32.557693 17462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:32.547727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:39175 (local address 127.0.0.1:47808)
0809 02:20:32.548203 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0809 02:20:32.548253 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 02:20:32.548380 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:20:32.548753 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:20:32.548760 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:20:32.549092 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:20:32.549274 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:20:32.549280 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:20:32.550217 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:20:32.550220 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:20:32.550967 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:20:32.550977 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:20:32.551640 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:20:32.551644 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:20:32.551799 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:32.552398 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:20:32.552417 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:20:32.553940 (+ 1523us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:20:32.555815 (+ 1875us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:20:32.555822 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:20:32.555825 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:20:32.556046 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:20:32.556352 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:20:32.556355 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:20:32.556357 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:20:32.556454 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:20:32.556929 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:20:32.556935 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:20:32.557052 (+ 117us) client_negotiation.cc:769] Sending connection context
0809 02:20:32.557281 (+ 229us) client_negotiation.cc:240] Negotiation successful
0809 02:20:32.557462 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":166,"threads_started":1}
I20260809 02:20:32.558704 17458 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.75.189" port: 42969 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 02:20:32 dist-test-slave-s11t krb5kdc[17121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786242031, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
Aug 09 02:20:32 dist-test-slave-s11t krb5kdc[17121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786242032, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.188@KRBTEST.COM for kudu/127.15.75.189@KRBTEST.COM
I20260809 02:20:32.566535 17463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:32.557044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47820 (local address 127.15.75.190:39175)
0809 02:20:32.557381 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:32.557391 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:32.557403 (+ 12us) server_negotiation.cc:407] Connection header received
0809 02:20:32.557441 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:32.557445 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:32.557486 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:32.557579 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:32.558515 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:32.559215 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:32.560276 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:32.560623 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:32.560798 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:32.563234 (+ 2436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:20:32.563237 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:20:32.563238 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:20:32.563259 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:20:32.564838 (+ 1579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:32.565315 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:32.565318 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:32.565319 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:32.565395 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:32.565688 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:32.565690 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:32.565691 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:32.565889 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:20:32.566087 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:32.566200 (+ 113us) server_negotiation.cc:299] Negotiation successful
0809 02:20:32.566332 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268,"thread_start_us":115,"threads_started":1}
I20260809 02:20:32.567528 17462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:32.559152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.189:42969 (local address 127.0.0.1:38230)
0809 02:20:32.559406 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0809 02:20:32.559420 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:20:32.559498 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:20:32.559720 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:20:32.559733 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:20:32.559904 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:20:32.560004 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:20:32.560009 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:20:32.561048 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:20:32.561053 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:20:32.561546 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:20:32.561553 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:20:32.562030 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:20:32.562034 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:20:32.562127 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:32.562540 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:20:32.562550 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:20:32.563901 (+ 1351us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:20:32.565904 (+ 2003us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:20:32.565908 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:20:32.565912 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:20:32.566158 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:20:32.566474 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:20:32.566479 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:20:32.566496 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:20:32.566569 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:20:32.567041 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:20:32.567044 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:20:32.567138 (+ 94us) client_negotiation.cc:769] Sending connection context
0809 02:20:32.567258 (+ 120us) client_negotiation.cc:240] Negotiation successful
0809 02:20:32.567368 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180}
I20260809 02:20:32.567600 17354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:32.559272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38230 (local address 127.15.75.189:42969)
0809 02:20:32.559405 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:32.559408 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:32.559462 (+ 54us) server_negotiation.cc:407] Connection header received
0809 02:20:32.559579 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:32.559581 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:32.559620 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:32.559679 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:32.560156 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:32.560944 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:32.561662 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:32.561918 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:32.562029 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:32.564084 (+ 2055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:20:32.564088 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:20:32.564092 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:20:32.564124 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:20:32.565804 (+ 1680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:32.566265 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:32.566270 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:32.566273 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:32.566359 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:32.566723 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:32.566726 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:32.566728 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:32.566942 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:20:32.567022 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:32.567330 (+ 308us) server_negotiation.cc:299] Negotiation successful
0809 02:20:32.567452 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 02:20:32.568650 17458 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.75.188" port: 43171 } has no permanent_uuid. Determining permanent_uuid...
I20260809 02:20:32.569607 17238 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.75.188" port: 43171 } attempt: 3
Aug 09 02:20:32 dist-test-slave-s11t krb5kdc[17121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786242031, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.75.189@KRBTEST.COM
I20260809 02:20:32.573441 17344 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.75.188" port: 43171 } attempt: 3
Aug 09 02:20:32 dist-test-slave-s11t krb5kdc[17121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786242032, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.188@KRBTEST.COM for kudu/127.15.75.188@KRBTEST.COM
Aug 09 02:20:32 dist-test-slave-s11t krb5kdc[17121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786242032, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.190@KRBTEST.COM for kudu/127.15.75.188@KRBTEST.COM
I20260809 02:20:32.577816 17461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:32.569249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35546 (local address 127.15.75.188:43171)
0809 02:20:32.569361 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:32.569365 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:32.569401 (+ 36us) server_negotiation.cc:407] Connection header received
0809 02:20:32.569431 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:32.569433 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:32.569469 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:32.569540 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:32.569918 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:32.571023 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:32.572121 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:32.572347 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:32.572424 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:32.574470 (+ 2046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:20:32.574473 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:20:32.574475 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:20:32.574496 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:20:32.576059 (+ 1563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:32.576587 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:32.576590 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:32.576592 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:32.576658 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:32.576992 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:32.576997 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:32.576999 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:32.577193 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:20:32.577267 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:32.577563 (+ 296us) server_negotiation.cc:299] Negotiation successful
0809 02:20:32.577665 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 02:20:32.578306 17462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:32.569187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.188:43171 (local address 127.0.0.1:35546)
0809 02:20:32.569268 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0809 02:20:32.569282 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:20:32.569351 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:20:32.569562 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:20:32.569565 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:20:32.569718 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:20:32.569797 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:20:32.569804 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:20:32.571140 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:20:32.571143 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:20:32.571988 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:20:32.572001 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:20:32.572468 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:20:32.572473 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:20:32.572553 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:32.573050 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:20:32.573065 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:20:32.574353 (+ 1288us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:20:32.576159 (+ 1806us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:20:32.576162 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:20:32.576164 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:20:32.576484 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:20:32.576778 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:20:32.576783 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:20:32.576787 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:20:32.576876 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:20:32.577324 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:20:32.577326 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:20:32.577399 (+ 73us) client_negotiation.cc:769] Sending connection context
0809 02:20:32.577989 (+ 590us) client_negotiation.cc:240] Negotiation successful
0809 02:20:32.578121 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
I20260809 02:20:32.578504 17354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:32.567707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38236 (local address 127.15.75.189:42969)
0809 02:20:32.567829 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:32.567832 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:32.567846 (+ 14us) server_negotiation.cc:407] Connection header received
0809 02:20:32.567967 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:32.567969 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:32.567998 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:32.568101 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:32.568910 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:32.569466 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:32.570485 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:32.570819 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:32.570916 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:32.574198 (+ 3282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:20:32.574201 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:20:32.574204 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:20:32.574226 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:20:32.575790 (+ 1564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:32.576660 (+ 870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:32.576664 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:32.576666 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:32.576775 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:32.577075 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:32.577078 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:32.577080 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:32.577281 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:20:32.577475 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:32.578248 (+ 773us) server_negotiation.cc:299] Negotiation successful
0809 02:20:32.578362 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":68}
Aug 09 02:20:32 dist-test-slave-s11t krb5kdc[17121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786242032, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.189@KRBTEST.COM for kudu/127.15.75.188@KRBTEST.COM
I20260809 02:20:32.581907 17458 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7da5ecc4ff624a17b8a738e7fdd07eb4: Bootstrap starting.
I20260809 02:20:32.582135 15662 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:20:32.582437 17243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:32.569971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.188:43171 (local address 127.0.0.1:35552)
0809 02:20:32.570102 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0809 02:20:32.570116 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:20:32.570183 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:20:32.572819 (+ 2636us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:20:32.572822 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:20:32.573017 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:20:32.573134 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:20:32.573142 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:20:32.574242 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:20:32.574246 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:20:32.575110 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:20:32.575124 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:20:32.575798 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:20:32.575803 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:20:32.575904 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:32.576366 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:20:32.576381 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:20:32.577764 (+ 1383us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:20:32.580945 (+ 3181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:20:32.580948 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:20:32.580950 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:20:32.581182 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:20:32.581503 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:20:32.581505 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:20:32.581507 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:20:32.581592 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:20:32.582006 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:20:32.582009 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:20:32.582097 (+ 88us) client_negotiation.cc:769] Sending connection context
0809 02:20:32.582174 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 02:20:32.582289 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260809 02:20:32.583045 17464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:32.571186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35552 (local address 127.15.75.188:43171)
0809 02:20:32.572572 (+ 1386us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:32.572581 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:32.572594 (+ 13us) server_negotiation.cc:407] Connection header received
0809 02:20:32.572629 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:32.572632 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:32.572676 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:32.572803 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:32.573272 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:32.574116 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:32.575258 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:32.575660 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:32.576112 (+ 452us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:32.577907 (+ 1795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:20:32.577912 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:20:32.577916 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:20:32.578053 (+ 137us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:20:32.580824 (+ 2771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:32.581300 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:32.581303 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:32.581304 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:32.581397 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:32.581703 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:32.581706 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:32.581708 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:32.581885 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:20:32.582716 (+ 831us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:32.582789 (+ 73us) server_negotiation.cc:299] Negotiation successful
0809 02:20:32.582892 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1308,"thread_start_us":92,"threads_started":1}
I20260809 02:20:32.584766 17458 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7da5ecc4ff624a17b8a738e7fdd07eb4: Neither blocks nor log segments found. Creating new log.
I20260809 02:20:32.583534 17198 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "cdf77404778242bcad0bec1b3345c6cc"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47820
I20260809 02:20:32.585707 17458 log.cc:824] T 00000000000000000000000000000000 P 7da5ecc4ff624a17b8a738e7fdd07eb4: Log is configured to *not* fsync() on all Append() calls
I20260809 02:20:32.586421 17349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:32.573787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.188:43171 (local address 127.0.0.1:35562)
0809 02:20:32.573912 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0809 02:20:32.573927 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:20:32.573995 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:20:32.576049 (+ 2054us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:20:32.576054 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:20:32.576229 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:20:32.576335 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:20:32.576344 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:20:32.577478 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:20:32.577481 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:20:32.578267 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:20:32.578275 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:20:32.578902 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:20:32.578907 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:20:32.579006 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:32.579390 (+ 384us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:20:32.579403 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:20:32.580984 (+ 1581us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:20:32.584783 (+ 3799us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:20:32.584787 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:20:32.584789 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:20:32.585052 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:20:32.585389 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:20:32.585392 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:20:32.585394 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:20:32.585495 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:20:32.585947 (+ 452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:20:32.585950 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:20:32.586046 (+ 96us) client_negotiation.cc:769] Sending connection context
0809 02:20:32.586150 (+ 104us) client_negotiation.cc:240] Negotiation successful
0809 02:20:32.586268 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260809 02:20:32.587330 17465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:32.573996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35562 (local address 127.15.75.188:43171)
0809 02:20:32.574600 (+ 604us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:32.574610 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:32.574943 (+ 333us) server_negotiation.cc:407] Connection header received
0809 02:20:32.574988 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:32.574994 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:32.575914 (+ 920us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:32.576454 (+ 540us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:32.576600 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:32.577367 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:32.578407 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:32.578760 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:32.579216 (+ 456us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:32.581420 (+ 2204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:20:32.581424 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:20:32.581426 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:20:32.581455 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:20:32.584640 (+ 3185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:32.585174 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:32.585177 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:32.585179 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:32.585258 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:32.585607 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:32.585610 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:32.585612 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:32.585805 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:20:32.585939 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:32.586921 (+ 982us) server_negotiation.cc:299] Negotiation successful
0809 02:20:32.587087 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":503,"thread_start_us":97,"threads_started":1}
I20260809 02:20:32.587670 17458 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7da5ecc4ff624a17b8a738e7fdd07eb4: No bootstrap required, opened a new log
I20260809 02:20:32.587721 17238 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cdf77404778242bcad0bec1b3345c6cc: Bootstrap starting.
I20260809 02:20:32.590233 17344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 531d186bc17e45ea82bdd08bde9d6292: Bootstrap starting.
I20260809 02:20:32.590266 17238 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cdf77404778242bcad0bec1b3345c6cc: Neither blocks nor log segments found. Creating new log.
I20260809 02:20:32.591038 17238 log.cc:824] T 00000000000000000000000000000000 P cdf77404778242bcad0bec1b3345c6cc: Log is configured to *not* fsync() on all Append() calls
I20260809 02:20:32.592309 17458 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7da5ecc4ff624a17b8a738e7fdd07eb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdf77404778242bcad0bec1b3345c6cc" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 39175 } } peers { permanent_uuid: "531d186bc17e45ea82bdd08bde9d6292" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 42969 } } peers { permanent_uuid: "7da5ecc4ff624a17b8a738e7fdd07eb4" member_type: VOTER last_known_addr { host: "127.15.75.188" port: 43171 } }
I20260809 02:20:32.592546 17458 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7da5ecc4ff624a17b8a738e7fdd07eb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7da5ecc4ff624a17b8a738e7fdd07eb4, State: Initialized, Role: FOLLOWER
I20260809 02:20:32.592792 17344 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 531d186bc17e45ea82bdd08bde9d6292: Neither blocks nor log segments found. Creating new log.
I20260809 02:20:32.592926 17458 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7da5ecc4ff624a17b8a738e7fdd07eb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdf77404778242bcad0bec1b3345c6cc" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 39175 } } peers { permanent_uuid: "531d186bc17e45ea82bdd08bde9d6292" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 42969 } } peers { permanent_uuid: "7da5ecc4ff624a17b8a738e7fdd07eb4" member_type: VOTER last_known_addr { host: "127.15.75.188" port: 43171 } }
I20260809 02:20:32.593046 17238 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cdf77404778242bcad0bec1b3345c6cc: No bootstrap required, opened a new log
I20260809 02:20:32.593516 17474 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7da5ecc4ff624a17b8a738e7fdd07eb4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdf77404778242bcad0bec1b3345c6cc" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 39175 } } peers { permanent_uuid: "531d186bc17e45ea82bdd08bde9d6292" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 42969 } } peers { permanent_uuid: "7da5ecc4ff624a17b8a738e7fdd07eb4" member_type: VOTER last_known_addr { host: "127.15.75.188" port: 43171 } } }
I20260809 02:20:32.593624 17344 log.cc:824] T 00000000000000000000000000000000 P 531d186bc17e45ea82bdd08bde9d6292: Log is configured to *not* fsync() on all Append() calls
I20260809 02:20:32.593741 17474 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7da5ecc4ff624a17b8a738e7fdd07eb4 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:20:32.594274 17458 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7da5ecc4ff624a17b8a738e7fdd07eb4 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:20:32.595811 17238 raft_consensus.cc:348] T 00000000000000000000000000000000 P cdf77404778242bcad0bec1b3345c6cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdf77404778242bcad0bec1b3345c6cc" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 39175 } } peers { permanent_uuid: "531d186bc17e45ea82bdd08bde9d6292" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 42969 } } peers { permanent_uuid: "7da5ecc4ff624a17b8a738e7fdd07eb4" member_type: VOTER last_known_addr { host: "127.15.75.188" port: 43171 } }
I20260809 02:20:32.595968 17238 raft_consensus.cc:729] T 00000000000000000000000000000000 P cdf77404778242bcad0bec1b3345c6cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdf77404778242bcad0bec1b3345c6cc, State: Initialized, Role: FOLLOWER
I20260809 02:20:32.596292 17238 consensus_queue.cc:260] T 00000000000000000000000000000000 P cdf77404778242bcad0bec1b3345c6cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdf77404778242bcad0bec1b3345c6cc" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 39175 } } peers { permanent_uuid: "531d186bc17e45ea82bdd08bde9d6292" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 42969 } } peers { permanent_uuid: "7da5ecc4ff624a17b8a738e7fdd07eb4" member_type: VOTER last_known_addr { host: "127.15.75.188" port: 43171 } }
I20260809 02:20:32.596985 17344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 531d186bc17e45ea82bdd08bde9d6292: No bootstrap required, opened a new log
I20260809 02:20:32.597599 17238 sys_catalog.cc:564] T 00000000000000000000000000000000 P cdf77404778242bcad0bec1b3345c6cc [sys.catalog]: configured and running, proceeding with master startup.
W20260809 02:20:32.599872 17493 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7da5ecc4ff624a17b8a738e7fdd07eb4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:20:32.599944 17493 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:20:32.600581 17494 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdf77404778242bcad0bec1b3345c6cc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdf77404778242bcad0bec1b3345c6cc" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 39175 } } peers { permanent_uuid: "531d186bc17e45ea82bdd08bde9d6292" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 42969 } } peers { permanent_uuid: "7da5ecc4ff624a17b8a738e7fdd07eb4" member_type: VOTER last_known_addr { host: "127.15.75.188" port: 43171 } } }
I20260809 02:20:32.600672 17494 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdf77404778242bcad0bec1b3345c6cc [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:20:32.600889 17344 raft_consensus.cc:348] T 00000000000000000000000000000000 P 531d186bc17e45ea82bdd08bde9d6292 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdf77404778242bcad0bec1b3345c6cc" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 39175 } } peers { permanent_uuid: "531d186bc17e45ea82bdd08bde9d6292" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 42969 } } peers { permanent_uuid: "7da5ecc4ff624a17b8a738e7fdd07eb4" member_type: VOTER last_known_addr { host: "127.15.75.188" port: 43171 } }
I20260809 02:20:32.601083 17344 raft_consensus.cc:729] T 00000000000000000000000000000000 P 531d186bc17e45ea82bdd08bde9d6292 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 531d186bc17e45ea82bdd08bde9d6292, State: Initialized, Role: FOLLOWER
I20260809 02:20:32.598197 17355 master_runner.cc:186] Error getting master registration for 127.15.75.190:39175: OK, instance_id { permanent_uuid: "cdf77404778242bcad0bec1b3345c6cc" instance_seqno: 1786242032098161 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 02:20:32.601419 17355 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 02:20:32.601356 17344 consensus_queue.cc:260] T 00000000000000000000000000000000 P 531d186bc17e45ea82bdd08bde9d6292 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdf77404778242bcad0bec1b3345c6cc" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 39175 } } peers { permanent_uuid: "531d186bc17e45ea82bdd08bde9d6292" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 42969 } } peers { permanent_uuid: "7da5ecc4ff624a17b8a738e7fdd07eb4" member_type: VOTER last_known_addr { host: "127.15.75.188" port: 43171 } }
I20260809 02:20:32.601580 17198 raft_consensus.cc:482] T 00000000000000000000000000000000 P cdf77404778242bcad0bec1b3345c6cc [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 02:20:32.601697 17198 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cdf77404778242bcad0bec1b3345c6cc [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:20:32.602591 17344 sys_catalog.cc:564] T 00000000000000000000000000000000 P 531d186bc17e45ea82bdd08bde9d6292 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:20:32.602959 17198 raft_consensus.cc:504] T 00000000000000000000000000000000 P cdf77404778242bcad0bec1b3345c6cc [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdf77404778242bcad0bec1b3345c6cc" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 39175 } } peers { permanent_uuid: "531d186bc17e45ea82bdd08bde9d6292" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 42969 } } peers { permanent_uuid: "7da5ecc4ff624a17b8a738e7fdd07eb4" member_type: VOTER last_known_addr { host: "127.15.75.188" port: 43171 } }
W20260809 02:20:32.603506 17514 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cdf77404778242bcad0bec1b3345c6cc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:20:32.603565 17514 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:20:32.603942 17198 leader_election.cc:288] T 00000000000000000000000000000000 P cdf77404778242bcad0bec1b3345c6cc [CANDIDATE]: Term 1 election: Requested vote from peers 531d186bc17e45ea82bdd08bde9d6292 (127.15.75.189:42969), 7da5ecc4ff624a17b8a738e7fdd07eb4 (127.15.75.188:43171)
I20260809 02:20:32.604408 17414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cdf77404778242bcad0bec1b3345c6cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "7da5ecc4ff624a17b8a738e7fdd07eb4"
I20260809 02:20:32.604620 17414 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7da5ecc4ff624a17b8a738e7fdd07eb4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:20:32.602172 17141 master_runner.cc:186] Error getting master registration for 127.15.75.189:42969: OK, instance_id { permanent_uuid: "531d186bc17e45ea82bdd08bde9d6292" instance_seqno: 1786242032316415 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 02:20:32.605412 17141 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 02:20:32.605906 17311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cdf77404778242bcad0bec1b3345c6cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "531d186bc17e45ea82bdd08bde9d6292"
I20260809 02:20:32.606110 17311 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 531d186bc17e45ea82bdd08bde9d6292 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:20:32.606134 17414 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7da5ecc4ff624a17b8a738e7fdd07eb4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cdf77404778242bcad0bec1b3345c6cc in term 1.
I20260809 02:20:32.606559 17164 leader_election.cc:302] T 00000000000000000000000000000000 P cdf77404778242bcad0bec1b3345c6cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7da5ecc4ff624a17b8a738e7fdd07eb4, cdf77404778242bcad0bec1b3345c6cc; no voters:
I20260809 02:20:32.606825 17494 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cdf77404778242bcad0bec1b3345c6cc [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:20:32.607647 17311 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 531d186bc17e45ea82bdd08bde9d6292 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cdf77404778242bcad0bec1b3345c6cc in term 1.
I20260809 02:20:32.608749 17494 raft_consensus.cc:686] T 00000000000000000000000000000000 P cdf77404778242bcad0bec1b3345c6cc [term 1 LEADER]: Becoming Leader. State: Replica: cdf77404778242bcad0bec1b3345c6cc, State: Running, Role: LEADER
I20260809 02:20:32.609087 17494 consensus_queue.cc:237] T 00000000000000000000000000000000 P cdf77404778242bcad0bec1b3345c6cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdf77404778242bcad0bec1b3345c6cc" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 39175 } } peers { permanent_uuid: "531d186bc17e45ea82bdd08bde9d6292" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 42969 } } peers { permanent_uuid: "7da5ecc4ff624a17b8a738e7fdd07eb4" member_type: VOTER last_known_addr { host: "127.15.75.188" port: 43171 } }
I20260809 02:20:32.612802 17535 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdf77404778242bcad0bec1b3345c6cc [sys.catalog]: SysCatalogTable state changed. Reason: New leader cdf77404778242bcad0bec1b3345c6cc. Latest consensus state: current_term: 1 leader_uuid: "cdf77404778242bcad0bec1b3345c6cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdf77404778242bcad0bec1b3345c6cc" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 39175 } } peers { permanent_uuid: "531d186bc17e45ea82bdd08bde9d6292" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 42969 } } peers { permanent_uuid: "7da5ecc4ff624a17b8a738e7fdd07eb4" member_type: VOTER last_known_addr { host: "127.15.75.188" port: 43171 } } }
I20260809 02:20:32.612931 17535 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdf77404778242bcad0bec1b3345c6cc [sys.catalog]: This master's current role is: LEADER
I20260809 02:20:32.619177 17354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:32.608923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38240 (local address 127.15.75.189:42969)
0809 02:20:32.609032 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:32.609037 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:32.609523 (+ 486us) server_negotiation.cc:407] Connection header received
0809 02:20:32.609657 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:32.609661 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:32.609711 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:32.609785 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:32.610664 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:32.611338 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:32.612475 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:32.612846 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:32.612955 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:32.615415 (+ 2460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:20:32.615419 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:20:32.615422 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:20:32.615449 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:20:32.617200 (+ 1751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:32.617710 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:32.617714 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:32.617716 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:32.617804 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:32.618145 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:32.618150 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:32.618152 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:32.618437 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:20:32.618544 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:32.618914 (+ 370us) server_negotiation.cc:299] Negotiation successful
0809 02:20:32.619028 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 02:20:32.622843 17465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:32.608985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35568 (local address 127.15.75.188:43171)
0809 02:20:32.609091 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:32.609095 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:32.609874 (+ 779us) server_negotiation.cc:407] Connection header received
0809 02:20:32.610065 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:32.610071 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:32.610124 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:32.610224 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:32.611044 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:32.611768 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:32.614462 (+ 2694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:32.614696 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:32.614808 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:32.618132 (+ 3324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:20:32.618136 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:20:32.618138 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:20:32.618164 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:20:32.620007 (+ 1843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:32.621457 (+ 1450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:32.621460 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:32.621462 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:32.621555 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:32.621857 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:32.621873 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:32.621875 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:32.622077 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:20:32.622205 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:32.622559 (+ 354us) server_negotiation.cc:299] Negotiation successful
0809 02:20:32.622675 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 02:20:32.624922 17515 sys_catalog.cc:455] T 00000000000000000000000000000000 P 531d186bc17e45ea82bdd08bde9d6292 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdf77404778242bcad0bec1b3345c6cc" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 39175 } } peers { permanent_uuid: "531d186bc17e45ea82bdd08bde9d6292" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 42969 } } peers { permanent_uuid: "7da5ecc4ff624a17b8a738e7fdd07eb4" member_type: VOTER last_known_addr { host: "127.15.75.188" port: 43171 } } }
I20260809 02:20:32.625052 17515 sys_catalog.cc:458] T 00000000000000000000000000000000 P 531d186bc17e45ea82bdd08bde9d6292 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:20:32.625595 17548 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:20:32.626626 17548 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:20:32.628810 17463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:32.608877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47834 (local address 127.15.75.190:39175)
0809 02:20:32.609115 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:32.609120 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:32.619688 (+ 10568us) server_negotiation.cc:407] Connection header received
0809 02:20:32.619859 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:32.619863 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:32.619908 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:32.619957 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:32.620741 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:32.621801 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:32.622936 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:32.623334 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:32.623416 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:32.625132 (+ 1716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:20:32.625136 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:20:32.625139 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:20:32.625166 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:20:32.626953 (+ 1787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:32.627454 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:32.627458 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:32.627460 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:32.627544 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:32.627852 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:32.627855 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:32.627857 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:32.628059 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:20:32.628150 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:32.628519 (+ 369us) server_negotiation.cc:299] Negotiation successful
0809 02:20:32.628638 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159}
W20260809 02:20:32.631851 17534 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 531d186bc17e45ea82bdd08bde9d6292: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:20:32.631932 17534 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:20:32.637681 17311 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 531d186bc17e45ea82bdd08bde9d6292 [term 1 FOLLOWER]: Refusing update from remote peer cdf77404778242bcad0bec1b3345c6cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:20:32.637740 17414 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7da5ecc4ff624a17b8a738e7fdd07eb4 [term 1 FOLLOWER]: Refusing update from remote peer cdf77404778242bcad0bec1b3345c6cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:20:32.638296 17535 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cdf77404778242bcad0bec1b3345c6cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "531d186bc17e45ea82bdd08bde9d6292" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 42969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:20:32.638475 17494 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cdf77404778242bcad0bec1b3345c6cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7da5ecc4ff624a17b8a738e7fdd07eb4" member_type: VOTER last_known_addr { host: "127.15.75.188" port: 43171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:20:32.642508 17515 sys_catalog.cc:455] T 00000000000000000000000000000000 P 531d186bc17e45ea82bdd08bde9d6292 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cdf77404778242bcad0bec1b3345c6cc. Latest consensus state: current_term: 1 leader_uuid: "cdf77404778242bcad0bec1b3345c6cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdf77404778242bcad0bec1b3345c6cc" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 39175 } } peers { permanent_uuid: "531d186bc17e45ea82bdd08bde9d6292" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 42969 } } peers { permanent_uuid: "7da5ecc4ff624a17b8a738e7fdd07eb4" member_type: VOTER last_known_addr { host: "127.15.75.188" port: 43171 } } }
I20260809 02:20:32.642612 17515 sys_catalog.cc:458] T 00000000000000000000000000000000 P 531d186bc17e45ea82bdd08bde9d6292 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:20:32.642540 17474 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7da5ecc4ff624a17b8a738e7fdd07eb4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cdf77404778242bcad0bec1b3345c6cc. Latest consensus state: current_term: 1 leader_uuid: "cdf77404778242bcad0bec1b3345c6cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdf77404778242bcad0bec1b3345c6cc" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 39175 } } peers { permanent_uuid: "531d186bc17e45ea82bdd08bde9d6292" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 42969 } } peers { permanent_uuid: "7da5ecc4ff624a17b8a738e7fdd07eb4" member_type: VOTER last_known_addr { host: "127.15.75.188" port: 43171 } } }
I20260809 02:20:32.642649 17474 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7da5ecc4ff624a17b8a738e7fdd07eb4 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:20:32.648783 17494 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdf77404778242bcad0bec1b3345c6cc [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cdf77404778242bcad0bec1b3345c6cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdf77404778242bcad0bec1b3345c6cc" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 39175 } } peers { permanent_uuid: "531d186bc17e45ea82bdd08bde9d6292" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 42969 } } peers { permanent_uuid: "7da5ecc4ff624a17b8a738e7fdd07eb4" member_type: VOTER last_known_addr { host: "127.15.75.188" port: 43171 } } }
I20260809 02:20:32.648898 17494 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdf77404778242bcad0bec1b3345c6cc [sys.catalog]: This master's current role is: LEADER
I20260809 02:20:32.649153 17550 mvcc.cc:205] Tried to move back new op lower bound from 7316447365675679744 to 7316447365570134016. Current Snapshot: MvccSnapshot[applied={T|T < 7316447365675679744}]
I20260809 02:20:32.649227 17515 sys_catalog.cc:455] T 00000000000000000000000000000000 P 531d186bc17e45ea82bdd08bde9d6292 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cdf77404778242bcad0bec1b3345c6cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdf77404778242bcad0bec1b3345c6cc" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 39175 } } peers { permanent_uuid: "531d186bc17e45ea82bdd08bde9d6292" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 42969 } } peers { permanent_uuid: "7da5ecc4ff624a17b8a738e7fdd07eb4" member_type: VOTER last_known_addr { host: "127.15.75.188" port: 43171 } } }
I20260809 02:20:32.649322 17515 sys_catalog.cc:458] T 00000000000000000000000000000000 P 531d186bc17e45ea82bdd08bde9d6292 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:20:32.650017 17551 mvcc.cc:205] Tried to move back new op lower bound from 7316447365675679744 to 7316447365570134016. Current Snapshot: MvccSnapshot[applied={T|T < 7316447365675679744}]
I20260809 02:20:32.652504 17535 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdf77404778242bcad0bec1b3345c6cc [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cdf77404778242bcad0bec1b3345c6cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdf77404778242bcad0bec1b3345c6cc" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 39175 } } peers { permanent_uuid: "531d186bc17e45ea82bdd08bde9d6292" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 42969 } } peers { permanent_uuid: "7da5ecc4ff624a17b8a738e7fdd07eb4" member_type: VOTER last_known_addr { host: "127.15.75.188" port: 43171 } } }
I20260809 02:20:32.652607 17535 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdf77404778242bcad0bec1b3345c6cc [sys.catalog]: This master's current role is: LEADER
I20260809 02:20:32.653546 17474 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7da5ecc4ff624a17b8a738e7fdd07eb4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cdf77404778242bcad0bec1b3345c6cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdf77404778242bcad0bec1b3345c6cc" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 39175 } } peers { permanent_uuid: "531d186bc17e45ea82bdd08bde9d6292" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 42969 } } peers { permanent_uuid: "7da5ecc4ff624a17b8a738e7fdd07eb4" member_type: VOTER last_known_addr { host: "127.15.75.188" port: 43171 } } }
I20260809 02:20:32.653646 17474 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7da5ecc4ff624a17b8a738e7fdd07eb4 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:20:32.653860 17552 mvcc.cc:205] Tried to move back new op lower bound from 7316447365675679744 to 7316447365570134016. Current Snapshot: MvccSnapshot[applied={T|T < 7316447365675679744}]
I20260809 02:20:32.657101 17548 catalog_manager.cc:1347] Generated new cluster ID: ac2ddd02c23140199d233390b7da4a81
I20260809 02:20:32.657158 17548 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:20:32.696784 17548 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:20:32.698165 17548 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:20:32.711689 17548 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cdf77404778242bcad0bec1b3345c6cc: Generated new TSK 0
I20260809 02:20:32.712621 17548 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 02:20:32.744441 17354 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:20:32.741939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38242 (local address 127.15.75.189:42969)
0809 02:20:32.742062 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:32.742067 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:32.743361 (+ 1294us) server_negotiation.cc:407] Connection header received
0809 02:20:32.743571 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:32.743575 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:32.743625 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:32.743678 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:32.744300 (+ 622us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38242: BlockingRecv error: recv got EOF from 127.0.0.1:38242 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260809 02:20:32.746140 17463 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:20:32.742735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47842 (local address 127.15.75.190:39175)
0809 02:20:32.742848 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:32.742852 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:32.745183 (+ 2331us) server_negotiation.cc:407] Connection header received
0809 02:20:32.745441 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:32.745444 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:32.745494 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:32.745546 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:32.746035 (+ 489us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47842: BlockingRecv error: recv got EOF from 127.0.0.1:47842 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260809 02:20:32.748159 17465 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:20:32.746881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35570 (local address 127.15.75.188:43171)
0809 02:20:32.746989 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:32.746993 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:32.747282 (+ 289us) server_negotiation.cc:407] Connection header received
0809 02:20:32.747465 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:32.747468 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:32.747514 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:32.747565 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:32.748052 (+ 487us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35570: BlockingRecv error: recv got EOF from 127.0.0.1:35570 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260809 02:20:32.752671 17354 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:20:32.751068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38244 (local address 127.15.75.189:42969)
0809 02:20:32.751167 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:32.751171 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:32.751742 (+ 571us) server_negotiation.cc:407] Connection header received
0809 02:20:32.751959 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:32.751962 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:32.752033 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:32.752083 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:32.752578 (+ 495us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38244: BlockingRecv error: recv got EOF from 127.0.0.1:38244 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260809 02:20:32.773553 17354 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:20:32.765900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38258 (local address 127.15.75.189:42969)
0809 02:20:32.766013 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:32.766018 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:32.772734 (+ 6716us) server_negotiation.cc:407] Connection header received
0809 02:20:32.772807 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:32.772810 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:32.772870 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:32.772975 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:32.773446 (+ 471us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38258: BlockingRecv error: recv got EOF from 127.0.0.1:38258 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260809 02:20:32.783818 17354 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:20:32.781259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38270 (local address 127.15.75.189:42969)
0809 02:20:32.781365 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:32.781369 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:32.782831 (+ 1462us) server_negotiation.cc:407] Connection header received
0809 02:20:32.783039 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:32.783043 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:32.783099 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:32.783157 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:32.783714 (+ 557us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38270: BlockingRecv error: recv got EOF from 127.0.0.1:38270 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260809 02:20:32.791446 17354 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:20:32.789961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38276 (local address 127.15.75.189:42969)
0809 02:20:32.790081 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:32.790085 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:32.790616 (+ 531us) server_negotiation.cc:407] Connection header received
0809 02:20:32.790800 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:32.790803 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:32.790845 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:32.790892 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:32.791355 (+ 463us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38276: BlockingRecv error: recv got EOF from 127.0.0.1:38276 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260809 02:20:32.803568 17354 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:20:32.801764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38278 (local address 127.15.75.189:42969)
0809 02:20:32.801870 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:32.801875 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:32.802421 (+ 546us) server_negotiation.cc:407] Connection header received
0809 02:20:32.802774 (+ 353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:32.802779 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:32.802843 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:32.802906 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:32.803473 (+ 567us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38278: BlockingRecv error: recv got EOF from 127.0.0.1:38278 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260809 02:20:32.823694 17354 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:20:32.822115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38280 (local address 127.15.75.189:42969)
0809 02:20:32.822254 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:32.822259 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:32.822808 (+ 549us) server_negotiation.cc:407] Connection header received
0809 02:20:32.822933 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:32.822937 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:32.822992 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:32.823083 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:32.823584 (+ 501us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38280: BlockingRecv error: recv got EOF from 127.0.0.1:38280 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260809 02:20:32.858731 17354 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:20:32.857542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38296 (local address 127.15.75.189:42969)
0809 02:20:32.857690 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:32.857696 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:32.857840 (+ 144us) server_negotiation.cc:407] Connection header received
0809 02:20:32.858019 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:32.858022 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:32.858077 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:32.858167 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:32.858630 (+ 463us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38296: BlockingRecv error: recv got EOF from 127.0.0.1:38296 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260809 02:20:32.925983 17354 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:20:32.924910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38298 (local address 127.15.75.189:42969)
0809 02:20:32.925044 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:32.925048 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:32.925195 (+ 147us) server_negotiation.cc:407] Connection header received
0809 02:20:32.925316 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:32.925319 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:32.925367 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:32.925453 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:32.925848 (+ 395us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38298: BlockingRecv error: recv got EOF from 127.0.0.1:38298 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260809 02:20:33.057447 17354 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:20:33.056194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38304 (local address 127.15.75.189:42969)
0809 02:20:33.056360 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:33.056365 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:33.056505 (+ 140us) server_negotiation.cc:407] Connection header received
0809 02:20:33.056708 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:33.056716 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:33.056767 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:33.056863 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:33.057330 (+ 467us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38304: BlockingRecv error: recv got EOF from 127.0.0.1:38304 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260809 02:20:33.317063 17354 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:20:33.315705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38320 (local address 127.15.75.189:42969)
0809 02:20:33.315849 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:33.315855 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:33.315962 (+ 107us) server_negotiation.cc:407] Connection header received
0809 02:20:33.316150 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:33.316153 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:33.316210 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:33.316304 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:33.316941 (+ 637us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38320: BlockingRecv error: recv got EOF from 127.0.0.1:38320 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 02:20:33.601262 17493 catalog_manager.cc:1259] Loaded cluster ID: ac2ddd02c23140199d233390b7da4a81
I20260809 02:20:33.601348 17493 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7da5ecc4ff624a17b8a738e7fdd07eb4: loading cluster ID for follower catalog manager: success
I20260809 02:20:33.603052 17493 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7da5ecc4ff624a17b8a738e7fdd07eb4: acquiring CA information for follower catalog manager: success
I20260809 02:20:33.604326 17493 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7da5ecc4ff624a17b8a738e7fdd07eb4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 02:20:33.633217 17534 catalog_manager.cc:1259] Loaded cluster ID: ac2ddd02c23140199d233390b7da4a81
I20260809 02:20:33.633294 17534 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 531d186bc17e45ea82bdd08bde9d6292: loading cluster ID for follower catalog manager: success
I20260809 02:20:33.634985 17534 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 531d186bc17e45ea82bdd08bde9d6292: acquiring CA information for follower catalog manager: success
I20260809 02:20:33.636132 17534 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 531d186bc17e45ea82bdd08bde9d6292: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 02:20:33.835286 17656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:33.831191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38334 (local address 127.15.75.189:42969)
0809 02:20:33.831464 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:33.831469 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:33.831578 (+ 109us) server_negotiation.cc:407] Connection header received
0809 02:20:33.831761 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:33.831765 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:33.831908 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:33.832032 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:20:33.832304 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:33.832871 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:33.833752 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:33.834004 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:33.834117 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:33.834455 (+ 338us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:20:33.834886 (+ 431us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:20:33.835029 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:33.835072 (+ 43us) server_negotiation.cc:299] Negotiation successful
0809 02:20:33.835127 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":75,"threads_started":1}
I20260809 02:20:33.841764 17667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:33.838415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35586 (local address 127.15.75.188:43171)
0809 02:20:33.838645 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:33.838649 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:33.838662 (+ 13us) server_negotiation.cc:407] Connection header received
0809 02:20:33.838739 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:33.838741 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:33.838843 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:33.838947 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:20:33.839149 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:33.839635 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:33.840457 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:33.840670 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:33.840805 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:33.841064 (+ 259us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:20:33.841419 (+ 355us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:20:33.841507 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:33.841607 (+ 100us) server_negotiation.cc:299] Negotiation successful
0809 02:20:33.841655 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":88,"threads_started":1}
I20260809 02:20:33.849157 17667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:33.844925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35592 (local address 127.15.75.188:43171)
0809 02:20:33.845013 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:33.845019 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:33.845514 (+ 495us) server_negotiation.cc:407] Connection header received
0809 02:20:33.845610 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:33.845615 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:33.845665 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:33.845731 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:20:33.845980 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:33.846570 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:33.848009 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:33.848247 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:33.848357 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:33.848625 (+ 268us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:20:33.848747 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:20:33.848821 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:33.848989 (+ 168us) server_negotiation.cc:299] Negotiation successful
0809 02:20:33.849044 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 02:20:33.849184 17656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:33.844801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38340 (local address 127.15.75.189:42969)
0809 02:20:33.844920 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:33.844925 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:33.845227 (+ 302us) server_negotiation.cc:407] Connection header received
0809 02:20:33.845438 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:33.845442 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:33.845504 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:33.845574 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:20:33.845863 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:33.846649 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:33.847991 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:33.848296 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:33.848393 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:33.848754 (+ 361us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:20:33.848833 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:20:33.848908 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:33.849029 (+ 121us) server_negotiation.cc:299] Negotiation successful
0809 02:20:33.849076 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 02:20:33.849849 17681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:33.844676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47846 (local address 127.15.75.190:39175)
0809 02:20:33.844976 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:33.844980 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:33.845174 (+ 194us) server_negotiation.cc:407] Connection header received
0809 02:20:33.845353 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:33.845360 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:33.845538 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:33.845652 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:20:33.846066 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:33.846846 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:33.847684 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:33.847957 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:33.848055 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:33.848457 (+ 402us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:20:33.848784 (+ 327us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:20:33.848868 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:33.849643 (+ 775us) server_negotiation.cc:299] Negotiation successful
0809 02:20:33.849703 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":75,"threads_started":1}
I20260809 02:20:33.852831 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 17141
I20260809 02:20:33.869609 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 17251
I20260809 02:20:33.879683 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 17355
2026-08-09T02:20:33Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3336 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241741788589-15662-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:20:33 dist-test-slave-s11t krb5kdc[17685](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:20:33 dist-test-slave-s11t krb5kdc[17685](info): set up 2 sockets
Aug 09 02:20:33 dist-test-slave-s11t krb5kdc[17685](info): commencing operation
krb5kdc: starting...
W20260809 02:20:35.113976 15662 mini_kdc.cc:121] Time spent starting KDC: real 1.207s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241741788589-15662-0/krb5kdc/test-user.keytab.
Aug 09 02:20:35 dist-test-slave-s11t krb5kdc[17685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786242035, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:20:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:20:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.75.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab.
Entry for principal kudu/127.15.75.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab.
I20260809 02:20:35.235301 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241741788589-15662-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241741788589-15662-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241741788589-15662-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:37291
--webserver_interface=127.15.75.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab
--principal=kudu/127.15.75.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:35481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.75.190:37291,127.15.75.189:46471,127.15.75.188:46189
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:20:35.344264 17702 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 02:20:35.344516 17702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:20:35.344539 17702 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:20:35.344574 17702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:20:35.348457 17702 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:20:35.348508 17702 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:20:35.348527 17702 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:20:35.348543 17702 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:20:35.348558 17702 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:20:35.353686 17702 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:35481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241741788589-15662-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241741788589-15662-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.75.190:37291,127.15.75.189:46471,127.15.75.188:46189
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.190.keytab
--principal=kudu/127.15.75.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.190:37291
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.75.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17702
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241741788589-15662-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:20:35.354665 17702 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:20:35.355615 17702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 02:20:35.364177 17702 server_base.cc:1034] running on GCE node
W20260809 02:20:35.364120 17712 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:20:35.364130 17709 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:20:35.364143 17708 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:20:35.365001 17702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:20:35.365971 17702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:20:35.367137 17702 hybrid_clock.cc:648] HybridClock initialized: now 1786242035367111 us; error 39 us; skew 500 ppm
Aug 09 02:20:35 dist-test-slave-s11t krb5kdc[17685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786242035, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:20:35.369801 17702 init.cc:377] Logged in from keytab as kudu/127.15.75.190@KRBTEST.COM (short username kudu)
I20260809 02:20:35.370651 17702 webserver.cc:460] Webserver started at http://127.15.75.190:40031/ using document root <none> and password file <none>
I20260809 02:20:35.371152 17702 fs_manager.cc:359] Metadata directory not provided
I20260809 02:20:35.371209 17702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:20:35.371361 17702 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:20:35.373296 17702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241741788589-15662-0/minicluster-data/master-0/data/instance:
uuid: "b557662d6a8a49d3bf148326b82456fa"
format_stamp: "Formatted at 2026-08-09 02:20:35 on dist-test-slave-s11t"
server_key: "6300fddeb7c9fdbb727876c529081250"
server_key_iv: "17fea84dc87cce1f72c848be26a78a0e"
server_key_version: "encryptionkey@0"
I20260809 02:20:35.373792 17702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241741788589-15662-0/minicluster-data/master-0/wal/instance:
uuid: "b557662d6a8a49d3bf148326b82456fa"
format_stamp: "Formatted at 2026-08-09 02:20:35 on dist-test-slave-s11t"
server_key: "6300fddeb7c9fdbb727876c529081250"
server_key_iv: "17fea84dc87cce1f72c848be26a78a0e"
server_key_version: "encryptionkey@0"
I20260809 02:20:35.377264 17702 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 02:20:35.379598 17721 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:20:35.380842 17702 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 02:20:35.380954 17702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241741788589-15662-0/minicluster-data/master-0/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241741788589-15662-0/minicluster-data/master-0/wal
uuid: "b557662d6a8a49d3bf148326b82456fa"
format_stamp: "Formatted at 2026-08-09 02:20:35 on dist-test-slave-s11t"
server_key: "6300fddeb7c9fdbb727876c529081250"
server_key_iv: "17fea84dc87cce1f72c848be26a78a0e"
server_key_version: "encryptionkey@0"
I20260809 02:20:35.381034 17702 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241741788589-15662-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241741788589-15662-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241741788589-15662-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:20:35.391695 17702 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:20:35.394466 17702 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:20:35.394608 17702 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:20:35.401321 17702 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.190:37291
I20260809 02:20:35.401618 17798 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.190:37291 every 8 connection(s)
I20260809 02:20:35.402233 17702 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241741788589-15662-0/minicluster-data/master-0/data/info.pb
I20260809 02:20:35.404898 17801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:20:35.408571 17801 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.75.190" port: 37291 } has no permanent_uuid. Determining permanent_uuid...
I20260809 02:20:35.411028 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 17702
I20260809 02:20:35.411154 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241741788589-15662-0/minicluster-data/master-0/wal/instance
I20260809 02:20:35.411437 15662 external_mini_cluster.cc:1428] Setting key 492ad7f49de3d79158525cef0322387a
Aug 09 02:20:35 dist-test-slave-s11t krb5kdc[17685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786242035, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.190@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.75.189@KRBTEST.COM; defaulting to no policy
I20260809 02:20:35.423980 17805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:35.410703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50918 (local address 127.15.75.190:37291)
0809 02:20:35.411129 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:35.411148 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:35.411445 (+ 297us) server_negotiation.cc:407] Connection header received
0809 02:20:35.412490 (+ 1045us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:35.412495 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:35.412766 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:35.412963 (+ 197us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:35.413963 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:35.414902 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:35.415863 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:35.416592 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:35.416848 (+ 256us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:35.419385 (+ 2537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:20:35.419394 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:20:35.419408 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:20:35.419438 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:20:35.421382 (+ 1944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:35.421825 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:35.421829 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:35.421831 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:35.421933 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:35.422243 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:35.422246 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:35.422247 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:35.422594 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:20:35.422714 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:35.423045 (+ 331us) server_negotiation.cc:299] Negotiation successful
0809 02:20:35.423227 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":45,"server-negotiator.queue_time_us":271,"thread_start_us":139,"threads_started":1}
I20260809 02:20:35.423985 17806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:35.410703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:37291 (local address 127.0.0.1:50918)
0809 02:20:35.411286 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0809 02:20:35.411334 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 02:20:35.412188 (+ 854us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:20:35.412963 (+ 775us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:20:35.412968 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:20:35.413426 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:20:35.413791 (+ 365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:20:35.413812 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:20:35.415034 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:20:35.415036 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:20:35.415740 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:20:35.415749 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:20:35.416799 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:20:35.416802 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:20:35.416977 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:35.417573 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:20:35.417588 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:20:35.419225 (+ 1637us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:20:35.421489 (+ 2264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:20:35.421497 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:20:35.421505 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:20:35.421721 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:20:35.422039 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:20:35.422044 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:20:35.422050 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:20:35.422159 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:20:35.422704 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:20:35.422709 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:20:35.422827 (+ 118us) client_negotiation.cc:769] Sending connection context
0809 02:20:35.422967 (+ 140us) client_negotiation.cc:240] Negotiation successful
0809 02:20:35.423307 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":412,"mutex_wait_us":168,"thread_start_us":132,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.189@KRBTEST.COM" created.
I20260809 02:20:35.426553 17801 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.75.189" port: 46471 } has no permanent_uuid. Determining permanent_uuid...
I20260809 02:20:35.427242 17806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:35.427004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.189:46471 (local address 127.0.0.1:60430)
0809 02:20:35.427116 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0809 02:20:35.427192 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.189:46471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260809 02:20:35.427513 17725 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.75.189:46471: connect: Connection refused (error 111)
I20260809 02:20:35.428325 17806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:35.428138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.189:46471 (local address 127.0.0.1:60444)
0809 02:20:35.428238 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0809 02:20:35.428279 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.189:46471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260809 02:20:35.428553 17801 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.75.189:46471: Network error: Client connection negotiation failed: client connection to 127.15.75.189:46471: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.189.keytab.
Entry for principal kudu/127.15.75.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.189.keytab.
I20260809 02:20:35.439477 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241741788589-15662-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241741788589-15662-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241741788589-15662-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241741788589-15662-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.189:46471
--webserver_interface=127.15.75.189
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.189.keytab
--principal=kudu/127.15.75.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:35481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.75.190:37291,127.15.75.189:46471,127.15.75.188:46189
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 02:20:35.477806 17801 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.75.189" port: 46471 } attempt: 1
I20260809 02:20:35.478724 17806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:35.478419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.189:46471 (local address 127.0.0.1:60454)
0809 02:20:35.478583 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 02:20:35.478647 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.189:46471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260809 02:20:35.479477 17806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:35.479250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.189:46471 (local address 127.0.0.1:60470)
0809 02:20:35.479362 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0809 02:20:35.479418 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.189:46471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260809 02:20:35.479607 17801 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.75.189:46471: Network error: Client connection negotiation failed: client connection to 127.15.75.189:46471: connect: Connection refused (error 111)
I20260809 02:20:35.547849 17801 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.75.189" port: 46471 } attempt: 2
I20260809 02:20:35.548751 17806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:35.548437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.189:46471 (local address 127.0.0.1:60478)
0809 02:20:35.548591 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 02:20:35.548653 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.189:46471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260809 02:20:35.549500 17806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:35.549280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.189:46471 (local address 127.0.0.1:60486)
0809 02:20:35.549389 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0809 02:20:35.549443 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.189:46471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260809 02:20:35.549628 17801 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.75.189:46471: Network error: Client connection negotiation failed: client connection to 127.15.75.189:46471: connect: Connection refused (error 111)
W20260809 02:20:35.550879 17814 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 02:20:35.551167 17814 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:20:35.551234 17814 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:20:35.551298 17814 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:20:35.555279 17814 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:20:35.555361 17814 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:20:35.555397 17814 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:20:35.555429 17814 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:20:35.555466 17814 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:20:35.560823 17814 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:35481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241741788589-15662-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241741788589-15662-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.15.75.190:37291,127.15.75.189:46471,127.15.75.188:46189
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.189.keytab
--principal=kudu/127.15.75.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.189:46471
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241741788589-15662-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.15.75.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17814
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241741788589-15662-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:20:35.561846 17814 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:20:35.562855 17814 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:20:35.571484 17823 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:20:35.571542 17825 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:20:35.571542 17827 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:20:35.571650 17814 server_base.cc:1034] running on GCE node
I20260809 02:20:35.572278 17814 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:20:35.573352 17814 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:20:35.574517 17814 hybrid_clock.cc:648] HybridClock initialized: now 1786242035574488 us; error 40 us; skew 500 ppm
Aug 09 02:20:35 dist-test-slave-s11t krb5kdc[17685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786242035, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:20:35.577220 17814 init.cc:377] Logged in from keytab as kudu/127.15.75.189@KRBTEST.COM (short username kudu)
I20260809 02:20:35.578122 17814 webserver.cc:460] Webserver started at http://127.15.75.189:34329/ using document root <none> and password file <none>
I20260809 02:20:35.578675 17814 fs_manager.cc:359] Metadata directory not provided
I20260809 02:20:35.578770 17814 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:20:35.578997 17814 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:20:35.581058 17814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241741788589-15662-0/minicluster-data/master-1/data/instance:
uuid: "8f384aa0d86342aa8837ce7cf929ff71"
format_stamp: "Formatted at 2026-08-09 02:20:35 on dist-test-slave-s11t"
server_key: "1ea7a72c5a6f44615cb49bccdcd0ddef"
server_key_iv: "42d4c09b3ed2e03f098498bd1d3d638f"
server_key_version: "encryptionkey@0"
I20260809 02:20:35.581621 17814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241741788589-15662-0/minicluster-data/master-1/wal/instance:
uuid: "8f384aa0d86342aa8837ce7cf929ff71"
format_stamp: "Formatted at 2026-08-09 02:20:35 on dist-test-slave-s11t"
server_key: "1ea7a72c5a6f44615cb49bccdcd0ddef"
server_key_iv: "42d4c09b3ed2e03f098498bd1d3d638f"
server_key_version: "encryptionkey@0"
I20260809 02:20:35.585309 17814 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 02:20:35.587621 17837 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:20:35.588850 17814 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.003s
I20260809 02:20:35.589005 17814 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241741788589-15662-0/minicluster-data/master-1/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241741788589-15662-0/minicluster-data/master-1/wal
uuid: "8f384aa0d86342aa8837ce7cf929ff71"
format_stamp: "Formatted at 2026-08-09 02:20:35 on dist-test-slave-s11t"
server_key: "1ea7a72c5a6f44615cb49bccdcd0ddef"
server_key_iv: "42d4c09b3ed2e03f098498bd1d3d638f"
server_key_version: "encryptionkey@0"
I20260809 02:20:35.589114 17814 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241741788589-15662-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241741788589-15662-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241741788589-15662-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:20:35.599861 17814 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:20:35.602535 17814 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:20:35.602739 17814 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:20:35.609970 17814 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.189:46471
I20260809 02:20:35.610289 17917 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.189:46471 every 8 connection(s)
I20260809 02:20:35.610913 17814 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241741788589-15662-0/minicluster-data/master-1/data/info.pb
I20260809 02:20:35.613765 17918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:20:35.615465 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 17814
I20260809 02:20:35.615590 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241741788589-15662-0/minicluster-data/master-1/wal/instance
I20260809 02:20:35.615896 15662 external_mini_cluster.cc:1428] Setting key 348d8d0670456e4b769eb1e6f6faf7c5
I20260809 02:20:35.618503 17918 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.75.190" port: 37291 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.15.75.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.75.188@KRBTEST.COM" created.
Aug 09 02:20:35 dist-test-slave-s11t krb5kdc[17685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786242035, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.189@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:20:35.632447 17805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:35.620298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50922 (local address 127.15.75.190:37291)
0809 02:20:35.620442 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:35.620446 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:35.621446 (+ 1000us) server_negotiation.cc:407] Connection header received
0809 02:20:35.622337 (+ 891us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:35.622341 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:35.622411 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:35.622513 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:35.623617 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:35.624163 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:35.625388 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:35.625764 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:35.625881 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:35.628585 (+ 2704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:20:35.628588 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:20:35.628590 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:20:35.628614 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:20:35.630124 (+ 1510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:35.630784 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:35.630787 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:35.630788 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:35.630869 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:35.631281 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:35.631284 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:35.631285 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:35.631458 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:20:35.631535 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:35.632212 (+ 677us) server_negotiation.cc:299] Negotiation successful
0809 02:20:35.632321 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 02:20:35.633661 17922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:35.620800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:37291 (local address 127.0.0.1:50922)
0809 02:20:35.621280 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0809 02:20:35.621321 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 02:20:35.622133 (+ 812us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:20:35.622678 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:20:35.622696 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:20:35.623030 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:20:35.623467 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:20:35.623480 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:20:35.624304 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:20:35.624308 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:20:35.625248 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:20:35.625258 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:20:35.625931 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:20:35.625935 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:20:35.626103 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:35.626633 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:20:35.626649 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:20:35.628411 (+ 1762us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:20:35.630259 (+ 1848us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:20:35.630268 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:20:35.630285 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:20:35.630671 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:20:35.630981 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:20:35.630987 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:20:35.630992 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:20:35.631182 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:20:35.631576 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:20:35.631585 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:20:35.631920 (+ 335us) client_negotiation.cc:769] Sending connection context
0809 02:20:35.632144 (+ 224us) client_negotiation.cc:240] Negotiation successful
0809 02:20:35.632509 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":131,"threads_started":1}
I20260809 02:20:35.635157 17918 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.75.189" port: 46471 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 02:20:35 dist-test-slave-s11t krb5kdc[17685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786242035, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.189@KRBTEST.COM for kudu/127.15.75.189@KRBTEST.COM
I20260809 02:20:35.640782 17801 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.75.189" port: 46471 } attempt: 3
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.75.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.188.keytab.
Entry for principal kudu/127.15.75.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.188.keytab.
I20260809 02:20:35.644008 15662 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
/tmp/dist-test-tasktcWNU7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241741788589-15662-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241741788589-15662-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241741788589-15662-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241741788589-15662-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.188:46189
--webserver_interface=127.15.75.188
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.188.keytab
--principal=kudu/127.15.75.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.75.148:35481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.75.190:37291,127.15.75.189:46471,127.15.75.188:46189
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241741788589-15662-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241741788589-15662-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241741788589-15662-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 02:20:35.645157 17922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:35.635691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.189:46471 (local address 127.0.0.1:60502)
0809 02:20:35.635881 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 02:20:35.635897 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 02:20:35.635979 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:20:35.636392 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:20:35.636405 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:20:35.636577 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:20:35.636730 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:20:35.636738 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:20:35.637842 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:20:35.637844 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:20:35.638568 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:20:35.638578 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:20:35.639232 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:20:35.639235 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:20:35.639305 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:35.639800 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:20:35.639820 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:20:35.641223 (+ 1403us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:20:35.643566 (+ 2343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:20:35.643569 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:20:35.643571 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:20:35.643760 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:20:35.644048 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:20:35.644052 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:20:35.644054 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:20:35.644145 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:20:35.644597 (+ 452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:20:35.644600 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:20:35.644761 (+ 161us) client_negotiation.cc:769] Sending connection context
0809 02:20:35.644873 (+ 112us) client_negotiation.cc:240] Negotiation successful
0809 02:20:35.644993 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106}
I20260809 02:20:35.646003 17924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:35.635775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60502 (local address 127.15.75.189:46471)
0809 02:20:35.636051 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:35.636061 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:35.636080 (+ 19us) server_negotiation.cc:407] Connection header received
0809 02:20:35.636116 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:35.636120 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:35.636257 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:35.636358 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:35.636845 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:35.637736 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:35.638672 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:35.639129 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:35.639258 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:35.641364 (+ 2106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:20:35.641374 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:20:35.641390 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:20:35.641429 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:20:35.643449 (+ 2020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:35.643871 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:35.643881 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:35.643882 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:35.643955 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:35.644247 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:35.644250 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:35.644252 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:35.644490 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:20:35.644634 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:35.644998 (+ 364us) server_negotiation.cc:299] Negotiation successful
0809 02:20:35.645168 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":94,"threads_started":1}
Aug 09 02:20:35 dist-test-slave-s11t krb5kdc[17685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786242035, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.190@KRBTEST.COM for kudu/127.15.75.189@KRBTEST.COM
I20260809 02:20:35.647953 17918 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.75.188" port: 46189 } has no permanent_uuid. Determining permanent_uuid...
I20260809 02:20:35.649724 17922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:35.648429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.188:46189 (local address 127.0.0.1:47676)
0809 02:20:35.648555 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0809 02:20:35.648675 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.188:46189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260809 02:20:35.650017 17838 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.75.188:46189: connect: Connection refused (error 111)
I20260809 02:20:35.650959 17922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:35.650742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.188:46189 (local address 127.0.0.1:47692)
0809 02:20:35.650872 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0809 02:20:35.650911 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.188:46189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
W20260809 02:20:35.651120 17918 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.75.188:46189: Network error: Client connection negotiation failed: client connection to 127.15.75.188:46189: connect: Connection refused (error 111)
I20260809 02:20:35.651456 17806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:35.641214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.189:46471 (local address 127.0.0.1:60514)
0809 02:20:35.641339 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0809 02:20:35.641354 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:20:35.641461 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:20:35.641879 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:20:35.641884 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:20:35.642096 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:20:35.642226 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:20:35.642235 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:20:35.643193 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:20:35.643197 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:20:35.644614 (+ 1417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:20:35.644630 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:20:35.645272 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:20:35.645278 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:20:35.645387 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:35.645855 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:20:35.645866 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:20:35.647155 (+ 1289us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:20:35.649637 (+ 2482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:20:35.649640 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:20:35.649641 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:20:35.649886 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:20:35.650216 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:20:35.650219 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:20:35.650220 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:20:35.650332 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:20:35.650783 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:20:35.650789 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:20:35.650914 (+ 125us) client_negotiation.cc:769] Sending connection context
0809 02:20:35.651041 (+ 127us) client_negotiation.cc:240] Negotiation successful
0809 02:20:35.651207 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260809 02:20:35.652170 17925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:35.641273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60514 (local address 127.15.75.189:46471)
0809 02:20:35.641613 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:35.641625 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:35.641640 (+ 15us) server_negotiation.cc:407] Connection header received
0809 02:20:35.641678 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:35.641684 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:35.641728 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:35.641821 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:35.642359 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:35.643084 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:35.644777 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:35.645122 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:35.645600 (+ 478us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:35.647323 (+ 1723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:20:35.647330 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:20:35.647336 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:20:35.647366 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:20:35.649525 (+ 2159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:35.650005 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:35.650015 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:35.650019 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:35.650113 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:35.650448 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:35.650452 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:35.650453 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:35.650663 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:20:35.650766 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:35.651809 (+ 1043us) server_negotiation.cc:299] Negotiation successful
0809 02:20:35.651943 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":123,"threads_started":1}
I20260809 02:20:35.652885 17801 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.75.188" port: 46189 } has no permanent_uuid. Determining permanent_uuid...
I20260809 02:20:35.653652 17806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:35.653336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.188:46189 (local address 127.0.0.1:47704)
0809 02:20:35.653494 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 02:20:35.653560 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.188:46189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260809 02:20:35.654459 17806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:35.654120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.188:46189 (local address 127.0.0.1:47716)
0809 02:20:35.654298 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 02:20:35.654361 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.188:46189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
W20260809 02:20:35.654604 17801 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.75.188:46189: Network error: Client connection negotiation failed: client connection to 127.15.75.188:46189: connect: Connection refused (error 111)
I20260809 02:20:35.685775 17801 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.75.188" port: 46189 } attempt: 1
I20260809 02:20:35.686658 17806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:35.686307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.188:46189 (local address 127.0.0.1:47726)
0809 02:20:35.686487 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 02:20:35.686562 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.188:46189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260809 02:20:35.687433 17806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:35.687174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.188:46189 (local address 127.0.0.1:47734)
0809 02:20:35.687298 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0809 02:20:35.687356 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.188:46189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260809 02:20:35.687587 17801 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.75.188:46189: Network error: Client connection negotiation failed: client connection to 127.15.75.188:46189: connect: Connection refused (error 111)
I20260809 02:20:35.700393 17918 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.75.188" port: 46189 } attempt: 1
I20260809 02:20:35.701182 17922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:35.700883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.188:46189 (local address 127.0.0.1:47748)
0809 02:20:35.701034 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 02:20:35.701100 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.188:46189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260809 02:20:35.701913 17922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:35.701683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.188:46189 (local address 127.0.0.1:47754)
0809 02:20:35.701797 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0809 02:20:35.701849 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.188:46189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260809 02:20:35.702056 17918 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.75.188:46189: Network error: Client connection negotiation failed: client connection to 127.15.75.188:46189: connect: Connection refused (error 111)
W20260809 02:20:35.758373 17926 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 02:20:35.758754 17926 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:20:35.758834 17926 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:20:35.758910 17926 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260809 02:20:35.762765 17801 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.75.188" port: 46189 } attempt: 2
W20260809 02:20:35.763589 17926 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
I20260809 02:20:35.763600 17806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:35.763257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.188:46189 (local address 127.0.0.1:47756)
0809 02:20:35.763432 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 02:20:35.763505 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.188:46189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260809 02:20:35.763652 17926 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:20:35.763685 17926 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:20:35.763708 17926 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:20:35.763747 17926 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:20:35.764319 17806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:35.764096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.188:46189 (local address 127.0.0.1:47766)
0809 02:20:35.764204 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0809 02:20:35.764255 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.188:46189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260809 02:20:35.764472 17801 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.75.188:46189: Network error: Client connection negotiation failed: client connection to 127.15.75.188:46189: connect: Connection refused (error 111)
I20260809 02:20:35.769146 17926 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.75.148:35481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241741788589-15662-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241741788589-15662-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.15.75.190:37291,127.15.75.189:46471,127.15.75.188:46189
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241741788589-15662-0/krb5kdc/kudu_127.15.75.188.keytab
--principal=kudu/127.15.75.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.75.188:46189
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241741788589-15662-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.15.75.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17926
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241741788589-15662-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:42:25 UTC on ccycloud.quasar-erdzyp.root.comops.site
build id 4575767
I20260809 02:20:35.770088 17926 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:20:35.770234 17918 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.75.188" port: 46189 } attempt: 2
I20260809 02:20:35.770910 17922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:35.770649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.188:46189 (local address 127.0.0.1:47770)
0809 02:20:35.770772 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0809 02:20:35.770832 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.188:46189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260809 02:20:35.771311 17926 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 02:20:35.771631 17922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:35.771395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.188:46189 (local address 127.0.0.1:47776)
0809 02:20:35.771517 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0809 02:20:35.771568 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.75.188:46189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260809 02:20:35.771764 17918 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.75.188:46189: Network error: Client connection negotiation failed: client connection to 127.15.75.188:46189: connect: Connection refused (error 111)
W20260809 02:20:35.779603 17936 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:20:35.779575 17935 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:20:35.779593 17939 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:20:35.779651 17926 server_base.cc:1034] running on GCE node
I20260809 02:20:35.780370 17926 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:20:35.781525 17926 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:20:35.782718 17926 hybrid_clock.cc:648] HybridClock initialized: now 1786242035782705 us; error 23 us; skew 500 ppm
Aug 09 02:20:35 dist-test-slave-s11t krb5kdc[17685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786242035, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:20:35.785495 17926 init.cc:377] Logged in from keytab as kudu/127.15.75.188@KRBTEST.COM (short username kudu)
I20260809 02:20:35.786305 17926 webserver.cc:460] Webserver started at http://127.15.75.188:36963/ using document root <none> and password file <none>
I20260809 02:20:35.786856 17926 fs_manager.cc:359] Metadata directory not provided
I20260809 02:20:35.786948 17926 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:20:35.787185 17926 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:20:35.789218 17926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241741788589-15662-0/minicluster-data/master-2/data/instance:
uuid: "4706f66d105c460dafeca31963a84d04"
format_stamp: "Formatted at 2026-08-09 02:20:35 on dist-test-slave-s11t"
server_key: "2c39ea87b957e1439187b42b7b55648d"
server_key_iv: "1dc1dbbabe6e2aa1db61a1aa5d1f6c27"
server_key_version: "encryptionkey@0"
I20260809 02:20:35.789777 17926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241741788589-15662-0/minicluster-data/master-2/wal/instance:
uuid: "4706f66d105c460dafeca31963a84d04"
format_stamp: "Formatted at 2026-08-09 02:20:35 on dist-test-slave-s11t"
server_key: "2c39ea87b957e1439187b42b7b55648d"
server_key_iv: "1dc1dbbabe6e2aa1db61a1aa5d1f6c27"
server_key_version: "encryptionkey@0"
I20260809 02:20:35.793464 17926 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 02:20:35.795732 17946 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:20:35.796921 17926 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260809 02:20:35.797055 17926 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241741788589-15662-0/minicluster-data/master-2/data,/tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241741788589-15662-0/minicluster-data/master-2/wal
uuid: "4706f66d105c460dafeca31963a84d04"
format_stamp: "Formatted at 2026-08-09 02:20:35 on dist-test-slave-s11t"
server_key: "2c39ea87b957e1439187b42b7b55648d"
server_key_iv: "1dc1dbbabe6e2aa1db61a1aa5d1f6c27"
server_key_version: "encryptionkey@0"
I20260809 02:20:35.797163 17926 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241741788589-15662-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241741788589-15662-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241741788589-15662-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:20:35.807364 17926 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:20:35.810519 17926 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:20:35.810719 17926 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:20:35.817597 17926 rpc_server.cc:307] RPC server started. Bound to: 127.15.75.188:46189
I20260809 02:20:35.817863 18021 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.75.188:46189 every 8 connection(s)
I20260809 02:20:35.818567 17926 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241741788589-15662-0/minicluster-data/master-2/data/info.pb
I20260809 02:20:35.821480 15662 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu as pid 17926
I20260809 02:20:35.821611 15662 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktcWNU7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241741788589-15662-0/minicluster-data/master-2/wal/instance
I20260809 02:20:35.821712 18022 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:20:35.821949 15662 external_mini_cluster.cc:1428] Setting key 0613c0ad937dcb69bbad9e01517f4ea7
I20260809 02:20:35.826913 18022 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.75.190" port: 37291 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 02:20:35 dist-test-slave-s11t krb5kdc[17685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786242035, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.75.188@KRBTEST.COM
Aug 09 02:20:35 dist-test-slave-s11t krb5kdc[17685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786242035, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.188@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
I20260809 02:20:35.837060 18027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:35.823226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47788 (local address 127.15.75.188:46189)
0809 02:20:35.823616 (+ 390us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:35.823635 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:35.823675 (+ 40us) server_negotiation.cc:407] Connection header received
0809 02:20:35.824571 (+ 896us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:35.824595 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:35.825047 (+ 452us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:35.825413 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:35.826086 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:35.827586 (+ 1500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:35.828499 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:35.829184 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:35.829374 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:35.832352 (+ 2978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:20:35.832357 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:20:35.832366 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:20:35.832391 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:20:35.834072 (+ 1681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:35.834531 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:35.834537 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:35.834545 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:35.834690 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:35.835004 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:35.835008 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:35.835010 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:35.835468 (+ 458us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:20:35.835648 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:35.835953 (+ 305us) server_negotiation.cc:299] Negotiation successful
0809 02:20:35.836233 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":253,"thread_start_us":106,"threads_started":1}
I20260809 02:20:35.838656 18029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:35.828743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.190:37291 (local address 127.0.0.1:50932)
0809 02:20:35.829159 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0809 02:20:35.829207 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 02:20:35.829332 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:20:35.829727 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:20:35.829736 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:20:35.830152 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:20:35.830283 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:20:35.830289 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:20:35.831350 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:20:35.831354 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:20:35.832351 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:20:35.832361 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:20:35.832982 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:20:35.832985 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:20:35.833069 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:35.833653 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:20:35.833665 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:20:35.835221 (+ 1556us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:20:35.837094 (+ 1873us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:20:35.837103 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:20:35.837108 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:20:35.837391 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:20:35.837656 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:20:35.837659 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:20:35.837661 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:20:35.837753 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:20:35.838138 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:20:35.838143 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:20:35.838267 (+ 124us) client_negotiation.cc:769] Sending connection context
0809 02:20:35.838383 (+ 116us) client_negotiation.cc:240] Negotiation successful
0809 02:20:35.838520 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":149,"threads_started":1}
I20260809 02:20:35.838745 17805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:35.828800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50932 (local address 127.15.75.190:37291)
0809 02:20:35.828953 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:35.828958 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:35.829296 (+ 338us) server_negotiation.cc:407] Connection header received
0809 02:20:35.829484 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:35.829488 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:35.829543 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:35.829644 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:35.830419 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:35.831208 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:35.832471 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:35.832855 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:35.832967 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:35.835350 (+ 2383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:20:35.835352 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:20:35.835354 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:20:35.835376 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:20:35.836987 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:35.837489 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:35.837492 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:35.837493 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:35.837566 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:35.837837 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:35.837839 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:35.837840 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:35.838006 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:20:35.838124 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:35.838485 (+ 361us) server_negotiation.cc:299] Negotiation successful
0809 02:20:35.838601 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 02:20:35.839757 18022 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.75.189" port: 46471 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 02:20:35 dist-test-slave-s11t krb5kdc[17685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786242035, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.75.190@KRBTEST.COM
Aug 09 02:20:35 dist-test-slave-s11t krb5kdc[17685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786242035, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.188@KRBTEST.COM for kudu/127.15.75.189@KRBTEST.COM
I20260809 02:20:35.848023 17805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:35.839575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50944 (local address 127.15.75.190:37291)
0809 02:20:35.839819 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:35.839822 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:35.839831 (+ 9us) server_negotiation.cc:407] Connection header received
0809 02:20:35.839857 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:35.839859 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:35.839921 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:35.840015 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:35.840739 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:35.841210 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:35.841963 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:35.842172 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:35.842277 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:35.844895 (+ 2618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:20:35.844900 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:20:35.844911 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:20:35.844952 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:20:35.846280 (+ 1328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:35.846715 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:35.846719 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:35.846721 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:35.846856 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:35.847155 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:35.847158 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:35.847160 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:35.847362 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:20:35.847589 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:35.847721 (+ 132us) server_negotiation.cc:299] Negotiation successful
0809 02:20:35.847827 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182}
I20260809 02:20:35.848518 18029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:35.840203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.189:46471 (local address 127.0.0.1:60520)
0809 02:20:35.840311 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0809 02:20:35.840324 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:20:35.840395 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:20:35.840739 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:20:35.840741 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:20:35.840912 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:20:35.840993 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:20:35.840998 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:20:35.841727 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:20:35.841732 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:20:35.842248 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:20:35.842256 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:20:35.842866 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:20:35.842870 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:20:35.842979 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:35.843336 (+ 357us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:20:35.843355 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:20:35.845253 (+ 1898us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:20:35.847027 (+ 1774us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:20:35.847032 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:20:35.847034 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:20:35.847228 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:20:35.847510 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:20:35.847513 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:20:35.847515 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:20:35.847600 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:20:35.848020 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:20:35.848024 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:20:35.848124 (+ 100us) client_negotiation.cc:769] Sending connection context
0809 02:20:35.848234 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 02:20:35.848357 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260809 02:20:35.848549 17925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:35.840366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60520 (local address 127.15.75.189:46471)
0809 02:20:35.840527 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:35.840530 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:35.840542 (+ 12us) server_negotiation.cc:407] Connection header received
0809 02:20:35.840581 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:35.840583 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:35.840620 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:35.840739 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:35.841113 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:35.841618 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:35.842369 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:35.842740 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:35.842852 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:35.845413 (+ 2561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:20:35.845417 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:20:35.845419 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:20:35.845450 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:20:35.846912 (+ 1462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:35.847328 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:35.847333 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:35.847334 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:35.847416 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:35.847690 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:35.847695 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:35.847697 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:35.847863 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:20:35.848011 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:35.848294 (+ 283us) server_negotiation.cc:299] Negotiation successful
0809 02:20:35.848407 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 02:20:35.849236 18022 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.75.188" port: 46189 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 02:20:35 dist-test-slave-s11t krb5kdc[17685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786242035, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.188@KRBTEST.COM for kudu/127.15.75.188@KRBTEST.COM
Aug 09 02:20:35 dist-test-slave-s11t krb5kdc[17685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786242035, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.75.189@KRBTEST.COM
I20260809 02:20:35.858158 18029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:35.849665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.188:46189 (local address 127.0.0.1:47790)
0809 02:20:35.849803 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 02:20:35.849816 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:20:35.849884 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:20:35.850196 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:20:35.850201 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:20:35.850360 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:20:35.850450 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:20:35.850459 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:20:35.851401 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:20:35.851413 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:20:35.852193 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:20:35.852203 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:20:35.852867 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:20:35.852871 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:20:35.852935 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:35.853450 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:20:35.853463 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:20:35.854670 (+ 1207us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:20:35.856733 (+ 2063us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:20:35.856738 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:20:35.856741 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:20:35.856933 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:20:35.857225 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:20:35.857230 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:20:35.857232 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:20:35.857309 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:20:35.857670 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:20:35.857672 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:20:35.857755 (+ 83us) client_negotiation.cc:769] Sending connection context
0809 02:20:35.857858 (+ 103us) client_negotiation.cc:240] Negotiation successful
0809 02:20:35.857995 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260809 02:20:35.858183 18027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:35.849848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47790 (local address 127.15.75.188:46189)
0809 02:20:35.849974 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:35.849979 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:35.849990 (+ 11us) server_negotiation.cc:407] Connection header received
0809 02:20:35.850029 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:35.850031 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:35.850065 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:35.850134 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:35.850558 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:35.851287 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:35.852317 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:35.852662 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:35.852778 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:35.854815 (+ 2037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:20:35.854820 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:20:35.854824 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:20:35.854853 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:20:35.856566 (+ 1713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:35.857029 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:35.857035 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:35.857040 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:35.857127 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:35.857411 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:35.857414 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:35.857416 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:35.857575 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:20:35.857679 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:35.857914 (+ 235us) server_negotiation.cc:299] Negotiation successful
0809 02:20:35.858051 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":36,"server-negotiator.queue_time_us":66}
I20260809 02:20:35.859149 17925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:35.849431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60530 (local address 127.15.75.189:46471)
0809 02:20:35.849614 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:35.849618 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:35.849630 (+ 12us) server_negotiation.cc:407] Connection header received
0809 02:20:35.849665 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:35.849668 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:35.849709 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:35.849789 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:35.850547 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:35.851291 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:35.852317 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:35.852644 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:35.852773 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:35.855336 (+ 2563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:20:35.855343 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:20:35.855346 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:20:35.855374 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:20:35.857004 (+ 1630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:35.857386 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:35.857391 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:35.857393 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:35.857475 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:35.857744 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:35.857747 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:35.857749 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:35.857924 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:20:35.858110 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:35.858831 (+ 721us) server_negotiation.cc:299] Negotiation successful
0809 02:20:35.858957 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260809 02:20:35.862244 18022 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4706f66d105c460dafeca31963a84d04: Bootstrap starting.
I20260809 02:20:35.862383 15662 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:20:35.862915 17918 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.75.188" port: 46189 } attempt: 3
I20260809 02:20:35.863605 17801 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.75.188" port: 46189 } attempt: 3
I20260809 02:20:35.863721 17765 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "b557662d6a8a49d3bf148326b82456fa"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50944
I20260809 02:20:35.865105 18022 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4706f66d105c460dafeca31963a84d04: Neither blocks nor log segments found. Creating new log.
I20260809 02:20:35.866015 18022 log.cc:824] T 00000000000000000000000000000000 P 4706f66d105c460dafeca31963a84d04: Log is configured to *not* fsync() on all Append() calls
Aug 09 02:20:35 dist-test-slave-s11t krb5kdc[17685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786242035, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.189@KRBTEST.COM for kudu/127.15.75.188@KRBTEST.COM
I20260809 02:20:35.868352 18022 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4706f66d105c460dafeca31963a84d04: No bootstrap required, opened a new log
Aug 09 02:20:35 dist-test-slave-s11t krb5kdc[17685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786242035, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.75.190@KRBTEST.COM for kudu/127.15.75.188@KRBTEST.COM
I20260809 02:20:35.872467 18027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:35.863341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47792 (local address 127.15.75.188:46189)
0809 02:20:35.863641 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:35.863645 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:35.863658 (+ 13us) server_negotiation.cc:407] Connection header received
0809 02:20:35.863690 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:35.863692 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:35.863811 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:35.864307 (+ 496us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:35.864386 (+ 79us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:35.865153 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:35.866200 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:35.866536 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:35.866637 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:35.868719 (+ 2082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:20:35.868726 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:20:35.868732 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:20:35.868762 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:20:35.870656 (+ 1894us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:35.871178 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:35.871182 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:35.871186 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:35.871265 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:35.871610 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:35.871613 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:35.871615 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:35.871827 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:20:35.871967 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:35.872237 (+ 270us) server_negotiation.cc:299] Negotiation successful
0809 02:20:35.872335 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":229}
I20260809 02:20:35.872475 17922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:35.863290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.188:46189 (local address 127.0.0.1:47792)
0809 02:20:35.863441 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 02:20:35.863453 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:20:35.863509 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:20:35.863941 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:20:35.863944 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:20:35.864111 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:20:35.864198 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:20:35.864205 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:20:35.865364 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:20:35.865369 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:20:35.866031 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:20:35.866041 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:20:35.866641 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:20:35.866645 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:20:35.866733 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:35.867182 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:20:35.867196 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:20:35.868557 (+ 1361us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:20:35.870757 (+ 2200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:20:35.870760 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:20:35.870762 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:20:35.871010 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:20:35.871375 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:20:35.871381 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:20:35.871384 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:20:35.871479 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:20:35.871977 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:20:35.871983 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:20:35.872073 (+ 90us) client_negotiation.cc:769] Sending connection context
0809 02:20:35.872186 (+ 113us) client_negotiation.cc:240] Negotiation successful
0809 02:20:35.872321 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90}
I20260809 02:20:35.872663 18022 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4706f66d105c460dafeca31963a84d04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b557662d6a8a49d3bf148326b82456fa" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 37291 } } peers { permanent_uuid: "8f384aa0d86342aa8837ce7cf929ff71" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 46471 } } peers { permanent_uuid: "4706f66d105c460dafeca31963a84d04" member_type: VOTER last_known_addr { host: "127.15.75.188" port: 46189 } }
I20260809 02:20:35.872941 18022 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4706f66d105c460dafeca31963a84d04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4706f66d105c460dafeca31963a84d04, State: Initialized, Role: FOLLOWER
I20260809 02:20:35.873307 18022 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4706f66d105c460dafeca31963a84d04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b557662d6a8a49d3bf148326b82456fa" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 37291 } } peers { permanent_uuid: "8f384aa0d86342aa8837ce7cf929ff71" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 46471 } } peers { permanent_uuid: "4706f66d105c460dafeca31963a84d04" member_type: VOTER last_known_addr { host: "127.15.75.188" port: 46189 } }
I20260809 02:20:35.874276 18035 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4706f66d105c460dafeca31963a84d04 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b557662d6a8a49d3bf148326b82456fa" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 37291 } } peers { permanent_uuid: "8f384aa0d86342aa8837ce7cf929ff71" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 46471 } } peers { permanent_uuid: "4706f66d105c460dafeca31963a84d04" member_type: VOTER last_known_addr { host: "127.15.75.188" port: 46189 } } }
I20260809 02:20:35.874576 18035 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4706f66d105c460dafeca31963a84d04 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:20:35.874751 18022 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4706f66d105c460dafeca31963a84d04 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:20:35.875311 17806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:35.863949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.75.188:46189 (local address 127.0.0.1:47798)
0809 02:20:35.864847 (+ 898us) negotiation.cc:107] Waiting for socket to connect
0809 02:20:35.864861 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:20:35.864944 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:20:35.865440 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:20:35.865443 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:20:35.865617 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:20:35.865707 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:20:35.865714 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:20:35.866688 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:20:35.866694 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:20:35.867806 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:20:35.867818 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:20:35.868511 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:20:35.868515 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:20:35.868584 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:35.869075 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:20:35.869091 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:20:35.870744 (+ 1653us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:20:35.873637 (+ 2893us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:20:35.873641 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:20:35.873642 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:20:35.873886 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:20:35.874271 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:20:35.874277 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:20:35.874279 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:20:35.874376 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:20:35.874840 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:20:35.874843 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:20:35.874934 (+ 91us) client_negotiation.cc:769] Sending connection context
0809 02:20:35.875017 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 02:20:35.875138 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":818}
I20260809 02:20:35.875607 17918 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f384aa0d86342aa8837ce7cf929ff71: Bootstrap starting.
I20260809 02:20:35.875949 18032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:35.864139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47798 (local address 127.15.75.188:46189)
0809 02:20:35.865177 (+ 1038us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:35.865188 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:35.865200 (+ 12us) server_negotiation.cc:407] Connection header received
0809 02:20:35.865232 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:35.865236 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:35.865277 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:35.865408 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:35.865831 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:35.866555 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:35.867963 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:35.868336 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:35.868432 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:35.870869 (+ 2437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:20:35.870872 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:20:35.870875 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:20:35.870909 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:20:35.873529 (+ 2620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:35.874006 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:35.874010 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:35.874011 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:35.874096 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:35.874513 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:35.874516 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:35.874517 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:35.874722 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:20:35.875581 (+ 859us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:35.875675 (+ 94us) server_negotiation.cc:299] Negotiation successful
0809 02:20:35.875784 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":970,"thread_start_us":94,"threads_started":1}
I20260809 02:20:35.878300 17918 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8f384aa0d86342aa8837ce7cf929ff71: Neither blocks nor log segments found. Creating new log.
I20260809 02:20:35.878333 17801 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b557662d6a8a49d3bf148326b82456fa: Bootstrap starting.
I20260809 02:20:35.879105 17918 log.cc:824] T 00000000000000000000000000000000 P 8f384aa0d86342aa8837ce7cf929ff71: Log is configured to *not* fsync() on all Append() calls
W20260809 02:20:35.879822 18049 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4706f66d105c460dafeca31963a84d04: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:20:35.879886 18049 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:20:35.880952 17918 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f384aa0d86342aa8837ce7cf929ff71: No bootstrap required, opened a new log
I20260809 02:20:35.881640 17801 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b557662d6a8a49d3bf148326b82456fa: Neither blocks nor log segments found. Creating new log.
I20260809 02:20:35.882560 17801 log.cc:824] T 00000000000000000000000000000000 P b557662d6a8a49d3bf148326b82456fa: Log is configured to *not* fsync() on all Append() calls
I20260809 02:20:35.884536 17801 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b557662d6a8a49d3bf148326b82456fa: No bootstrap required, opened a new log
I20260809 02:20:35.884907 17918 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8f384aa0d86342aa8837ce7cf929ff71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b557662d6a8a49d3bf148326b82456fa" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 37291 } } peers { permanent_uuid: "8f384aa0d86342aa8837ce7cf929ff71" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 46471 } } peers { permanent_uuid: "4706f66d105c460dafeca31963a84d04" member_type: VOTER last_known_addr { host: "127.15.75.188" port: 46189 } }
I20260809 02:20:35.885106 17918 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8f384aa0d86342aa8837ce7cf929ff71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f384aa0d86342aa8837ce7cf929ff71, State: Initialized, Role: FOLLOWER
I20260809 02:20:35.885407 17918 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f384aa0d86342aa8837ce7cf929ff71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b557662d6a8a49d3bf148326b82456fa" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 37291 } } peers { permanent_uuid: "8f384aa0d86342aa8837ce7cf929ff71" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 46471 } } peers { permanent_uuid: "4706f66d105c460dafeca31963a84d04" member_type: VOTER last_known_addr { host: "127.15.75.188" port: 46189 } }
I20260809 02:20:35.882879 17926 master_runner.cc:186] Error getting master registration for 127.15.75.190:37291: OK, instance_id { permanent_uuid: "b557662d6a8a49d3bf148326b82456fa" instance_seqno: 1786242035399458 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 02:20:35.886242 17926 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 02:20:35.886686 17918 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8f384aa0d86342aa8837ce7cf929ff71 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:20:35.887449 17801 raft_consensus.cc:348] T 00000000000000000000000000000000 P b557662d6a8a49d3bf148326b82456fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b557662d6a8a49d3bf148326b82456fa" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 37291 } } peers { permanent_uuid: "8f384aa0d86342aa8837ce7cf929ff71" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 46471 } } peers { permanent_uuid: "4706f66d105c460dafeca31963a84d04" member_type: VOTER last_known_addr { host: "127.15.75.188" port: 46189 } }
I20260809 02:20:35.887655 17801 raft_consensus.cc:729] T 00000000000000000000000000000000 P b557662d6a8a49d3bf148326b82456fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b557662d6a8a49d3bf148326b82456fa, State: Initialized, Role: FOLLOWER
I20260809 02:20:35.887957 17801 consensus_queue.cc:260] T 00000000000000000000000000000000 P b557662d6a8a49d3bf148326b82456fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b557662d6a8a49d3bf148326b82456fa" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 37291 } } peers { permanent_uuid: "8f384aa0d86342aa8837ce7cf929ff71" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 46471 } } peers { permanent_uuid: "4706f66d105c460dafeca31963a84d04" member_type: VOTER last_known_addr { host: "127.15.75.188" port: 46189 } }
I20260809 02:20:35.888530 18070 sys_catalog.cc:455] T 00000000000000000000000000000000 P b557662d6a8a49d3bf148326b82456fa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b557662d6a8a49d3bf148326b82456fa" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 37291 } } peers { permanent_uuid: "8f384aa0d86342aa8837ce7cf929ff71" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 46471 } } peers { permanent_uuid: "4706f66d105c460dafeca31963a84d04" member_type: VOTER last_known_addr { host: "127.15.75.188" port: 46189 } } }
I20260809 02:20:35.888798 18070 sys_catalog.cc:458] T 00000000000000000000000000000000 P b557662d6a8a49d3bf148326b82456fa [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:20:35.889586 17801 sys_catalog.cc:564] T 00000000000000000000000000000000 P b557662d6a8a49d3bf148326b82456fa [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:20:35.889686 17765 raft_consensus.cc:482] T 00000000000000000000000000000000 P b557662d6a8a49d3bf148326b82456fa [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 02:20:35.889810 17765 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b557662d6a8a49d3bf148326b82456fa [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:20:35.891049 17765 raft_consensus.cc:504] T 00000000000000000000000000000000 P b557662d6a8a49d3bf148326b82456fa [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b557662d6a8a49d3bf148326b82456fa" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 37291 } } peers { permanent_uuid: "8f384aa0d86342aa8837ce7cf929ff71" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 46471 } } peers { permanent_uuid: "4706f66d105c460dafeca31963a84d04" member_type: VOTER last_known_addr { host: "127.15.75.188" port: 46189 } }
I20260809 02:20:35.891922 17765 leader_election.cc:288] T 00000000000000000000000000000000 P b557662d6a8a49d3bf148326b82456fa [CANDIDATE]: Term 1 election: Requested vote from peers 8f384aa0d86342aa8837ce7cf929ff71 (127.15.75.189:46471), 4706f66d105c460dafeca31963a84d04 (127.15.75.188:46189)
I20260809 02:20:35.892818 17979 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b557662d6a8a49d3bf148326b82456fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "4706f66d105c460dafeca31963a84d04"
I20260809 02:20:35.893042 17979 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4706f66d105c460dafeca31963a84d04 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:20:35.890290 17814 master_runner.cc:186] Error getting master registration for 127.15.75.190:37291: OK, instance_id { permanent_uuid: "b557662d6a8a49d3bf148326b82456fa" instance_seqno: 1786242035399458 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 02:20:35.893661 17814 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 02:20:35.894413 17979 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4706f66d105c460dafeca31963a84d04 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b557662d6a8a49d3bf148326b82456fa in term 1.
W20260809 02:20:35.895373 18094 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b557662d6a8a49d3bf148326b82456fa: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260809 02:20:35.895431 17724 leader_election.cc:302] T 00000000000000000000000000000000 P b557662d6a8a49d3bf148326b82456fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4706f66d105c460dafeca31963a84d04, b557662d6a8a49d3bf148326b82456fa; no voters:
W20260809 02:20:35.895509 18094 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:20:35.895722 18070 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b557662d6a8a49d3bf148326b82456fa [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:20:35.895704 17875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b557662d6a8a49d3bf148326b82456fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "8f384aa0d86342aa8837ce7cf929ff71"
I20260809 02:20:35.895936 18070 raft_consensus.cc:686] T 00000000000000000000000000000000 P b557662d6a8a49d3bf148326b82456fa [term 1 LEADER]: Becoming Leader. State: Replica: b557662d6a8a49d3bf148326b82456fa, State: Running, Role: LEADER
I20260809 02:20:35.895934 17875 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8f384aa0d86342aa8837ce7cf929ff71 [term 0 FOLLOWER]: Advancing to term 1
W20260809 02:20:35.895946 17731 tablet.cc:2369] T 00000000000000000000000000000000 P b557662d6a8a49d3bf148326b82456fa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:20:35.896273 18070 consensus_queue.cc:237] T 00000000000000000000000000000000 P b557662d6a8a49d3bf148326b82456fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b557662d6a8a49d3bf148326b82456fa" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 37291 } } peers { permanent_uuid: "8f384aa0d86342aa8837ce7cf929ff71" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 46471 } } peers { permanent_uuid: "4706f66d105c460dafeca31963a84d04" member_type: VOTER last_known_addr { host: "127.15.75.188" port: 46189 } }
I20260809 02:20:35.897204 17875 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8f384aa0d86342aa8837ce7cf929ff71 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b557662d6a8a49d3bf148326b82456fa in term 1.
I20260809 02:20:35.898010 18070 sys_catalog.cc:455] T 00000000000000000000000000000000 P b557662d6a8a49d3bf148326b82456fa [sys.catalog]: SysCatalogTable state changed. Reason: New leader b557662d6a8a49d3bf148326b82456fa. Latest consensus state: current_term: 1 leader_uuid: "b557662d6a8a49d3bf148326b82456fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b557662d6a8a49d3bf148326b82456fa" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 37291 } } peers { permanent_uuid: "8f384aa0d86342aa8837ce7cf929ff71" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 46471 } } peers { permanent_uuid: "4706f66d105c460dafeca31963a84d04" member_type: VOTER last_known_addr { host: "127.15.75.188" port: 46189 } } }
I20260809 02:20:35.898111 18070 sys_catalog.cc:458] T 00000000000000000000000000000000 P b557662d6a8a49d3bf148326b82456fa [sys.catalog]: This master's current role is: LEADER
W20260809 02:20:35.900868 18079 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8f384aa0d86342aa8837ce7cf929ff71: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:20:35.900942 18079 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:20:35.902315 17702 master_runner.cc:263] No leader master found from master b557662d6a8a49d3bf148326b82456fa
I20260809 02:20:35.902384 17702 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 02:20:35.908972 18058 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f384aa0d86342aa8837ce7cf929ff71 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b557662d6a8a49d3bf148326b82456fa" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 37291 } } peers { permanent_uuid: "8f384aa0d86342aa8837ce7cf929ff71" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 46471 } } peers { permanent_uuid: "4706f66d105c460dafeca31963a84d04" member_type: VOTER last_known_addr { host: "127.15.75.188" port: 46189 } } }
I20260809 02:20:35.909231 18058 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f384aa0d86342aa8837ce7cf929ff71 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:20:35.910887 18107 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:20:35.911798 18107 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:20:35.912561 17925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:35.902319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60546 (local address 127.15.75.189:46471)
0809 02:20:35.902434 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:35.902440 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:35.902803 (+ 363us) server_negotiation.cc:407] Connection header received
0809 02:20:35.902945 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:35.902949 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:35.902996 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:35.903070 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:35.903904 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:35.904627 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:35.906049 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:35.906382 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:35.906458 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:35.908715 (+ 2257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:20:35.908719 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:20:35.908722 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:20:35.908748 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:20:35.910643 (+ 1895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:35.911146 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:35.911149 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:35.911151 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:35.911232 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:35.911557 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:35.911560 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:35.911562 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:35.911785 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:20:35.911888 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:35.912249 (+ 361us) server_negotiation.cc:299] Negotiation successful
0809 02:20:35.912364 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 02:20:35.912817 18027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:35.901100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47812 (local address 127.15.75.188:46189)
0809 02:20:35.901233 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:35.901237 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:35.902041 (+ 804us) server_negotiation.cc:407] Connection header received
0809 02:20:35.902627 (+ 586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:35.902631 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:35.902676 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:35.902776 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:35.903783 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:35.904584 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:35.906049 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:35.906382 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:35.906458 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:35.908715 (+ 2257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:20:35.908719 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:20:35.908722 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:20:35.908748 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:20:35.911019 (+ 2271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:35.911492 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:35.911499 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:35.911500 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:35.911585 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:35.911888 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:35.911891 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:35.911907 (+ 16us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:35.912109 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:20:35.912208 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:35.912541 (+ 333us) server_negotiation.cc:299] Negotiation successful
0809 02:20:35.912659 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 02:20:35.917387 17979 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4706f66d105c460dafeca31963a84d04 [term 1 FOLLOWER]: Refusing update from remote peer b557662d6a8a49d3bf148326b82456fa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:20:35.917474 17875 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8f384aa0d86342aa8837ce7cf929ff71 [term 1 FOLLOWER]: Refusing update from remote peer b557662d6a8a49d3bf148326b82456fa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:20:35.918056 18105 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b557662d6a8a49d3bf148326b82456fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f384aa0d86342aa8837ce7cf929ff71" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 46471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:20:35.918571 18105 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b557662d6a8a49d3bf148326b82456fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "4706f66d105c460dafeca31963a84d04" member_type: VOTER last_known_addr { host: "127.15.75.188" port: 46189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:20:35.921905 18035 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4706f66d105c460dafeca31963a84d04 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b557662d6a8a49d3bf148326b82456fa. Latest consensus state: current_term: 1 leader_uuid: "b557662d6a8a49d3bf148326b82456fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b557662d6a8a49d3bf148326b82456fa" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 37291 } } peers { permanent_uuid: "8f384aa0d86342aa8837ce7cf929ff71" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 46471 } } peers { permanent_uuid: "4706f66d105c460dafeca31963a84d04" member_type: VOTER last_known_addr { host: "127.15.75.188" port: 46189 } } }
I20260809 02:20:35.921947 18058 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f384aa0d86342aa8837ce7cf929ff71 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b557662d6a8a49d3bf148326b82456fa. Latest consensus state: current_term: 1 leader_uuid: "b557662d6a8a49d3bf148326b82456fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b557662d6a8a49d3bf148326b82456fa" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 37291 } } peers { permanent_uuid: "8f384aa0d86342aa8837ce7cf929ff71" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 46471 } } peers { permanent_uuid: "4706f66d105c460dafeca31963a84d04" member_type: VOTER last_known_addr { host: "127.15.75.188" port: 46189 } } }
I20260809 02:20:35.922031 18035 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4706f66d105c460dafeca31963a84d04 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:20:35.922049 18058 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f384aa0d86342aa8837ce7cf929ff71 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:20:35.923442 17805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:35.902108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50954 (local address 127.15.75.190:37291)
0809 02:20:35.902215 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:35.902219 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:35.903112 (+ 893us) server_negotiation.cc:407] Connection header received
0809 02:20:35.903289 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:35.903292 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:35.903337 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:35.903387 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:35.904793 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:35.905590 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:35.913096 (+ 7506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:35.913411 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:35.913491 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:35.915925 (+ 2434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:20:35.915928 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:20:35.915931 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:20:35.915956 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:20:35.918349 (+ 2393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:35.922151 (+ 3802us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:35.922154 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:35.922156 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:35.922245 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:20:35.922567 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:20:35.922571 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:20:35.922573 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:20:35.922773 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:20:35.922921 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:35.923169 (+ 248us) server_negotiation.cc:299] Negotiation successful
0809 02:20:35.923287 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 02:20:35.935237 18105 sys_catalog.cc:455] T 00000000000000000000000000000000 P b557662d6a8a49d3bf148326b82456fa [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b557662d6a8a49d3bf148326b82456fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b557662d6a8a49d3bf148326b82456fa" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 37291 } } peers { permanent_uuid: "8f384aa0d86342aa8837ce7cf929ff71" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 46471 } } peers { permanent_uuid: "4706f66d105c460dafeca31963a84d04" member_type: VOTER last_known_addr { host: "127.15.75.188" port: 46189 } } }
I20260809 02:20:35.935356 18105 sys_catalog.cc:458] T 00000000000000000000000000000000 P b557662d6a8a49d3bf148326b82456fa [sys.catalog]: This master's current role is: LEADER
I20260809 02:20:35.935465 18117 mvcc.cc:205] Tried to move back new op lower bound from 7316447379111370752 to 7316447379034554368. Current Snapshot: MvccSnapshot[applied={T|T < 7316447379111370752}]
I20260809 02:20:35.936085 18070 sys_catalog.cc:455] T 00000000000000000000000000000000 P b557662d6a8a49d3bf148326b82456fa [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b557662d6a8a49d3bf148326b82456fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b557662d6a8a49d3bf148326b82456fa" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 37291 } } peers { permanent_uuid: "8f384aa0d86342aa8837ce7cf929ff71" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 46471 } } peers { permanent_uuid: "4706f66d105c460dafeca31963a84d04" member_type: VOTER last_known_addr { host: "127.15.75.188" port: 46189 } } }
I20260809 02:20:35.936188 18070 sys_catalog.cc:458] T 00000000000000000000000000000000 P b557662d6a8a49d3bf148326b82456fa [sys.catalog]: This master's current role is: LEADER
I20260809 02:20:35.936709 18035 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4706f66d105c460dafeca31963a84d04 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b557662d6a8a49d3bf148326b82456fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b557662d6a8a49d3bf148326b82456fa" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 37291 } } peers { permanent_uuid: "8f384aa0d86342aa8837ce7cf929ff71" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 46471 } } peers { permanent_uuid: "4706f66d105c460dafeca31963a84d04" member_type: VOTER last_known_addr { host: "127.15.75.188" port: 46189 } } }
I20260809 02:20:35.936821 18035 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4706f66d105c460dafeca31963a84d04 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:20:35.937166 18058 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f384aa0d86342aa8837ce7cf929ff71 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b557662d6a8a49d3bf148326b82456fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b557662d6a8a49d3bf148326b82456fa" member_type: VOTER last_known_addr { host: "127.15.75.190" port: 37291 } } peers { permanent_uuid: "8f384aa0d86342aa8837ce7cf929ff71" member_type: VOTER last_known_addr { host: "127.15.75.189" port: 46471 } } peers { permanent_uuid: "4706f66d105c460dafeca31963a84d04" member_type: VOTER last_known_addr { host: "127.15.75.188" port: 46189 } } }
I20260809 02:20:35.937269 18058 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f384aa0d86342aa8837ce7cf929ff71 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:20:35.939635 18119 mvcc.cc:205] Tried to move back new op lower bound from 7316447379111370752 to 7316447379034554368. Current Snapshot: MvccSnapshot[applied={T|T < 7316447379111370752}]
I20260809 02:20:35.939823 18120 mvcc.cc:205] Tried to move back new op lower bound from 7316447379111370752 to 7316447379034554368. Current Snapshot: MvccSnapshot[applied={T|T < 7316447379111370752}]
I20260809 02:20:35.942159 18107 catalog_manager.cc:1347] Generated new cluster ID: 1d1955b71c8a4281baf3ac58f9c06a2d
I20260809 02:20:35.942226 18107 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:20:35.994860 18107 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:20:35.996145 18107 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:20:36.010389 18107 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b557662d6a8a49d3bf148326b82456fa: Generated new TSK 0
I20260809 02:20:36.011293 18107 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 02:20:36.048645 17805 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:20:36.045309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50958 (local address 127.15.75.190:37291)
0809 02:20:36.045436 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:36.045441 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:36.047514 (+ 2073us) server_negotiation.cc:407] Connection header received
0809 02:20:36.047726 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:36.047730 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:36.047785 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:36.047845 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:36.048516 (+ 671us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50958: BlockingRecv error: recv got EOF from 127.0.0.1:50958 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260809 02:20:36.050117 17925 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:20:36.046173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60554 (local address 127.15.75.189:46471)
0809 02:20:36.046315 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:36.046320 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:36.049189 (+ 2869us) server_negotiation.cc:407] Connection header received
0809 02:20:36.049383 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:36.049389 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:36.049441 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:36.049493 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:36.049993 (+ 500us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60554: BlockingRecv error: recv got EOF from 127.0.0.1:60554 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260809 02:20:36.051673 18027 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:20:36.046930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47828 (local address 127.15.75.188:46189)
0809 02:20:36.047065 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:36.047070 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:36.050678 (+ 3608us) server_negotiation.cc:407] Connection header received
0809 02:20:36.050913 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:36.050916 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:36.050966 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:36.051016 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:36.051552 (+ 536us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47828: BlockingRecv error: recv got EOF from 127.0.0.1:47828 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260809 02:20:36.055687 17925 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:20:36.054158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60556 (local address 127.15.75.189:46471)
0809 02:20:36.054285 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:36.054289 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:36.054751 (+ 462us) server_negotiation.cc:407] Connection header received
0809 02:20:36.054969 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:36.054972 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:36.055043 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:36.055098 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:36.055590 (+ 492us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60556: BlockingRecv error: recv got EOF from 127.0.0.1:60556 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260809 02:20:36.060809 17925 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:20:36.059328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60572 (local address 127.15.75.189:46471)
0809 02:20:36.059429 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:36.059432 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:36.059932 (+ 500us) server_negotiation.cc:407] Connection header received
0809 02:20:36.060112 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:36.060115 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:36.060162 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:36.060210 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:36.060715 (+ 505us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60572: BlockingRecv error: recv got EOF from 127.0.0.1:60572 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260809 02:20:36.066521 17925 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:20:36.064993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60584 (local address 127.15.75.189:46471)
0809 02:20:36.065104 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:36.065108 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:36.065661 (+ 553us) server_negotiation.cc:407] Connection header received
0809 02:20:36.065844 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:36.065848 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:36.065893 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:36.065942 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:36.066428 (+ 486us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60584: BlockingRecv error: recv got EOF from 127.0.0.1:60584 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260809 02:20:36.074425 17925 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:20:36.072749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60596 (local address 127.15.75.189:46471)
0809 02:20:36.072895 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:36.072899 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:36.073445 (+ 546us) server_negotiation.cc:407] Connection header received
0809 02:20:36.073647 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:36.073651 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:36.073702 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:36.073755 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:36.074329 (+ 574us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60596: BlockingRecv error: recv got EOF from 127.0.0.1:60596 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
W20260809 02:20:36.090364 17925 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:20:36.085068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60612 (local address 127.15.75.189:46471)
0809 02:20:36.085180 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:36.085184 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:36.089384 (+ 4200us) server_negotiation.cc:407] Connection header received
0809 02:20:36.089591 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:36.089594 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:36.089645 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:36.089696 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:36.090236 (+ 540us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60612: BlockingRecv error: recv got EOF from 127.0.0.1:60612 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260809 02:20:36.110474 17925 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:20:36.108780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60614 (local address 127.15.75.189:46471)
0809 02:20:36.108893 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:36.108897 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:36.109464 (+ 567us) server_negotiation.cc:407] Connection header received
0809 02:20:36.109657 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:36.109660 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:36.109715 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:36.109767 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:36.110347 (+ 580us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60614: BlockingRecv error: recv got EOF from 127.0.0.1:60614 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260809 02:20:36.146205 17925 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:20:36.145024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60616 (local address 127.15.75.189:46471)
0809 02:20:36.145146 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:36.145150 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:36.145388 (+ 238us) server_negotiation.cc:407] Connection header received
0809 02:20:36.145509 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:36.145511 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:36.145561 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:36.145634 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:36.146078 (+ 444us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60616: BlockingRecv error: recv got EOF from 127.0.0.1:60616 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260809 02:20:36.213328 17925 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:20:36.212271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60622 (local address 127.15.75.189:46471)
0809 02:20:36.212407 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:36.212411 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:36.212551 (+ 140us) server_negotiation.cc:407] Connection header received
0809 02:20:36.212669 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:36.212672 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:36.212743 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:36.212818 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:36.213232 (+ 414us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60622: BlockingRecv error: recv got EOF from 127.0.0.1:60622 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260809 02:20:36.344732 17925 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:20:36.343623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60638 (local address 127.15.75.189:46471)
0809 02:20:36.343769 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:36.343773 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:36.343940 (+ 167us) server_negotiation.cc:407] Connection header received
0809 02:20:36.344024 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:36.344027 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:36.344075 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:36.344156 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:36.344583 (+ 427us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60638: BlockingRecv error: recv got EOF from 127.0.0.1:60638 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260809 02:20:36.604194 17925 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:20:36.603158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60654 (local address 127.15.75.189:46471)
0809 02:20:36.603333 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:36.603337 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:36.603356 (+ 19us) server_negotiation.cc:407] Connection header received
0809 02:20:36.603515 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:36.603518 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:36.603570 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:36.603657 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:20:36.604092 (+ 435us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60654: BlockingRecv error: recv got EOF from 127.0.0.1:60654 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
I20260809 02:20:36.881244 18049 catalog_manager.cc:1259] Loaded cluster ID: 1d1955b71c8a4281baf3ac58f9c06a2d
I20260809 02:20:36.881331 18049 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4706f66d105c460dafeca31963a84d04: loading cluster ID for follower catalog manager: success
I20260809 02:20:36.883024 18049 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4706f66d105c460dafeca31963a84d04: acquiring CA information for follower catalog manager: success
I20260809 02:20:36.884263 18049 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4706f66d105c460dafeca31963a84d04: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 02:20:36.902158 18079 catalog_manager.cc:1259] Loaded cluster ID: 1d1955b71c8a4281baf3ac58f9c06a2d
I20260809 02:20:36.902220 18079 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8f384aa0d86342aa8837ce7cf929ff71: loading cluster ID for follower catalog manager: success
I20260809 02:20:36.903791 18079 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8f384aa0d86342aa8837ce7cf929ff71: acquiring CA information for follower catalog manager: success
I20260809 02:20:36.904915 18079 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8f384aa0d86342aa8837ce7cf929ff71: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 02:20:37.122610 18219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:37.118361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60658 (local address 127.15.75.189:46471)
0809 02:20:37.118624 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:37.118630 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:37.118714 (+ 84us) server_negotiation.cc:407] Connection header received
0809 02:20:37.118910 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:37.118914 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:37.119053 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:37.119180 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:20:37.119455 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:37.119995 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:37.120967 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:37.121224 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:37.121342 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:37.121713 (+ 371us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:20:37.122192 (+ 479us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:20:37.122340 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:37.122389 (+ 49us) server_negotiation.cc:299] Negotiation successful
0809 02:20:37.122445 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":80,"threads_started":1}
I20260809 02:20:37.129556 18236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:37.125890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47838 (local address 127.15.75.188:46189)
0809 02:20:37.126126 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:37.126131 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:37.126146 (+ 15us) server_negotiation.cc:407] Connection header received
0809 02:20:37.126251 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:37.126254 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:37.126374 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:37.126505 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:20:37.126725 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:37.127237 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:37.128065 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:37.128299 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:37.128416 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:37.128757 (+ 341us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:20:37.129141 (+ 384us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:20:37.129271 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:37.129336 (+ 65us) server_negotiation.cc:299] Negotiation successful
0809 02:20:37.129388 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":64,"threads_started":1}
I20260809 02:20:37.136267 18236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:37.132367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47850 (local address 127.15.75.188:46189)
0809 02:20:37.132491 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:37.132495 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:37.132981 (+ 486us) server_negotiation.cc:407] Connection header received
0809 02:20:37.133161 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:37.133166 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:37.133228 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:37.133275 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:20:37.133638 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:37.134150 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:37.135108 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:37.135426 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:37.135525 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:37.135812 (+ 287us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:20:37.135904 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:20:37.135975 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:37.136067 (+ 92us) server_negotiation.cc:299] Negotiation successful
0809 02:20:37.136120 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 02:20:37.136420 18219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:37.132480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60668 (local address 127.15.75.189:46471)
0809 02:20:37.132598 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:37.132602 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:37.132616 (+ 14us) server_negotiation.cc:407] Connection header received
0809 02:20:37.132752 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:37.132755 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:37.132806 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:37.132883 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:20:37.133261 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:37.134020 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:37.135229 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:37.135562 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:37.135649 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:37.135945 (+ 296us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:20:37.136021 (+ 76us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:20:37.136066 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:37.136235 (+ 169us) server_negotiation.cc:299] Negotiation successful
0809 02:20:37.136286 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 02:20:37.137171 18245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:20:37.132374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50966 (local address 127.15.75.190:37291)
0809 02:20:37.132625 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0809 02:20:37.132630 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:20:37.132870 (+ 240us) server_negotiation.cc:407] Connection header received
0809 02:20:37.133064 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:20:37.133066 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:20:37.133171 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:20:37.133308 (+ 137us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:20:37.133516 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:37.134280 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:37.135396 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:20:37.135780 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:20:37.135872 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:20:37.136184 (+ 312us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:20:37.136501 (+ 317us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:20:37.136596 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 02:20:37.136963 (+ 367us) server_negotiation.cc:299] Negotiation successful
0809 02:20:37.137030 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":80,"threads_started":1}
I20260809 02:20:37.140057 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 17702
I20260809 02:20:37.158679 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 17814
I20260809 02:20:37.170389 15662 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktcWNU7/build/debug/bin/kudu with pid 17926
2026-08-09T02:20:37Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3289 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6626 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (295378 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST